Two years ago you had two options. Now you have three. Open-weight models from DeepSeek, Qwen, Llama, and Mistral have reached the point where they’re competitive with proprietary APIs on most enterprise workloads. And a managed open-weight path now gives you API-like simplicity at open-weight model costs. The result is three distinct options — each with different cost structures, operational burdens, and risk profiles.
Most guidance still treats this as a binary choice: build or buy, proprietary or open. That framing isn’t useful anymore. The real question is which of these three deployment options fits which workload. Here’s the framework. This article is part of our comprehensive open-source AI model options series — for context on why the open-weight landscape changed so rapidly, that piece covers the competitive shift in detail.
Why is the build-vs-buy question more complex than it used to be?
Two years ago, open-weight models were too far behind proprietary models to be a serious option for most businesses. That gap has closed fast. DeepSeek V3 was trained for $5.6 million using 2,000 NVIDIA H800 GPUs, compared to $80–100 million and 16,000 H100 GPUs for comparable Western models — and it delivered competitive results. Qwen3 and Llama 3.x have followed the same trajectory.
The cost picture has also shifted. Proprietary API pricing has dropped, but open-weight inference costs have fallen further. Apache 2.0 licensing on the leading open-weight models — Llama, Qwen, Mistral, DeepSeek — removes the legal barriers to commercial deployment. Three viable options now exist. Which one is right for you depends on your workload, not your preference.
What are the three deployment options for open-weight AI?
Here’s the framework. Three options, each with a distinct trade-off profile.
Option A — Proprietary API (OpenAI, Anthropic Claude, Google Gemini)
Pay-per-token access to vendor-hosted models. No infrastructure to manage. Fastest time to value. But data leaves your environment on every API call, model versions change without notice, and per-token costs add up quickly at scale. Deep integration with a single provider’s API, prompt format, and feature set creates vendor lock-in that makes migration expensive when pricing or terms change.
Option B — Self-Hosted Open-Weight (vLLM + Llama/Qwen on your own GPUs)
You download the model weights and run inference on GPU hardware you control. Maximum data sovereignty. Lowest per-token cost at high volume. But the operational burden is entirely yours — CUDA stack management, GPU infrastructure, MLOps capacity, on-call rotation. vLLM supports over 100 model architectures and vendor-neutral hardware, which helps, but this option demands serious platform engineering expertise to run reliably.
Option C — Managed Open-Weight Service (AWS Bedrock, Azure AI Foundry, Groq, Baseten)
Open-weight model weights hosted by a cloud provider behind a managed API. You get open-weight model flexibility with API-like operational simplicity. No GPU infrastructure to manage. AWS Bedrock now offers nearly 100 serverless models, including Qwen3, DeepSeek V3.2, Mistral, and Llama variants. Cost sits between Option A and B. Lock-in is lower than proprietary APIs because the model weights are portable — if your provider changes pricing, you can move.
The sovereignty spectrum runs from Option A (lowest control) through Option C to Option B (highest control). Compliance requirements can eliminate options from the left before cost analysis even begins.
The recommended path for most teams: start with Option A to validate the workload, migrate to Option C at medium volume, and evaluate Option B when token spend and MLOps capacity both justify it. For a detailed look at AWS Bedrock as the managed middle path, Option C deserves its own treatment.
Which option fits which workload?
Make this decision per workload, not company-wide. Cost, latency, compliance, and predictability profiles all differ enough that a single approach rarely makes sense across all your AI usage.
RAG pipelines (document retrieval and generation): High-volume, often compliance-sensitive in HealthTech and FinTech. Option C (managed open-weight) is the recommended default. Option B (self-hosted) becomes viable at high steady-state volume. Option A only makes sense for low-volume proof-of-concept.
Coding agents: Qwen3 Coder Next achieves competitive SWE-Bench Pro scores, matching or outperforming models with far larger active compute. DeepSeek V3 matches or exceeds GPT-4o on code tasks. Option C is the recommended default here. For a model-by-model comparison, see evaluating open-weight models for coding agents.
Agentic workflows (multi-step tool use): This is where the open-weight advantage is highest. Long context runs combined with tool calls generate large token volumes that make proprietary APIs expensive at scale. Agentic architecture costs can make each transaction more expensive than the human process it was meant to replace if API pricing wasn’t factored into the pilot budget. Self-hosted or managed open-weight is the right call here.
Batch inference (nightly classification, enrichment): Predictable, schedulable, low latency requirements. Self-hosting is most cost-effective at volume. Managed open-weight works well for teams without GPU infrastructure.
Customer-facing chat: Highest availability and moderation requirements. Proprietary API may still be warranted for flagship customer experience due to SLA guarantees. Managed open-weight is acceptable when provider SLAs are sufficient.
One thing worth flagging: published benchmark scores often don’t reflect production workload characteristics. Validate your model selection on your own data before making routing decisions.
What does each option actually cost?
Total cost of ownership (TCO) is the right comparison unit. That means all direct costs — API spend or hardware — plus all indirect costs: engineering time, MLOps staffing, on-call burden, CUDA maintenance, staff turnover risk. Build-vs-buy analyses that compare only hardware cost against API spend miss most of the picture.
Option A TCO: Token consumption plus engineering time for integration and prompt maintenance. GPT-5 runs $1.25 input / $10 output per million tokens; Claude Sonnet 4 at $3 input / $15 output; Gemini 2.5 Flash at $0.30 input / $2.50 output. Cost scales with usage and becomes unpredictable with poorly designed applications.
Option B TCO: GPU hardware amortisation plus MLOps staffing plus on-call and CUDA maintenance. Enterprise AI specialists cost $200K–$500K+ annually — and that’s before you factor in hardware. Most organisations underestimate this because indirect costs get left out of the calculation.
Option C TCO: Per-token charges well below proprietary pricing — Groq’s Llama 3.3 70B at $0.59/$0.79 per million tokens; Qwen 2.5-Max at approximately $0.38/million tokens compared to Claude Sonnet 4 at $15/million output tokens. No infrastructure staffing required.
Break-even thresholds: At around 10M tokens/month, managed open-weight typically beats proprietary API on cost. Self-hosting becomes cost-competitive at 50M–100M tokens/month when MLOps staffing already exists. For teams in the 10M–50M token/month range, Option C is the natural fit.
What team capabilities does each option require?
Team capability is a hard constraint. Self-hosting without MLOps capacity is a reliability and security risk — it’s not just an efficiency issue.
Option A requires software engineers with API integration experience. No ML or infrastructure expertise needed.
Option B requires at minimum one dedicated MLOps or ML platform engineer. Not a senior backend engineer picking it up on the side. CUDA expertise, GPU infrastructure management, inference framework operation, model versioning, and on-call capacity are all required. A realistic minimum is a team of 8–10 engineers with one dedicated to ML platform work. These are specialisations that don’t come from general backend experience.
Option C requires backend engineers with cloud API integration skills and basic model selection knowledge. If your team has already built tooling against the OpenAI API format, migration to AWS Bedrock’s OpenAI-compatible endpoint via Project Mantle is straightforward.
For production AI, three distinct roles need to be covered: an AI product owner, an ML engineer, and a platform engineer. Option B requires all three. Options A and C can work with the first two.
The practical self-assessment for Option B: does your team have someone who has debugged a CUDA out-of-memory error in production? If the answer is no, Option B isn’t ready.
What are the risk factors for each option?
Risk differs across three dimensions — vendor risk, operational risk, and compliance risk — and each option trades off differently across all three.
Option A risks: Vendor lock-in occurs when source code and data become tied to a vendor’s environment, making migration costly. Custom prompt engineering, feature dependencies, and re-testing costs all compound the switching cost. Model versions change without notice. A single-vendor outage means a product outage.
Option B risks: Operational fragility. CUDA stack updates can break inference, GPU failures require rapid response, and MLOps expertise is scarce and expensive to replace. Model version control complexity grows without governance infrastructure — compliance and governance for open-weight AI in production covers this in depth.
Option C risks: Lower across most dimensions. Cloud provider lock-in is weaker than proprietary API lock-in because model weights are portable. SLA depends on provider reliability. Pricing is subject to change.
A few compliance considerations that act as hard filters. HIPAA PHI data cannot transit a proprietary API without a Business Associate Agreement, and not all providers offer BAAs. GDPR personal data must be processed within the EU or under an adequacy decision — this eliminates some US-only managed services for EU customer data. For a full governance treatment, see risk factors for open-weight AI in production.
Lock-in mitigation: Options B and C both preserve model weight portability. Moving from C to B is feasible. Moving from A to B or C requires an API abstraction layer.
How do you build an AI operating model that survives past the pilot phase?
Most AI pilots don’t fail because the model underperforms. They fail because there’s no operating model — no clear ownership, no version control, no cost monitoring, no escalation path. Nearly 95% of generative AI pilots never reach production scale, cycling through continuous testing without ever getting there.
The Databricks Enterprise AI Operating Model provides the organisational framing. Designate an AI product owner for each workload, an ML engineer for model selection, and a platform engineer for deployment. In small teams one person may cover multiple roles — but all three responsibilities must be assigned.
Run Options A, C, and eventually B simultaneously, with different workloads on different tiers based on volume, compliance, and cost. Two trigger conditions worth formalising: when monthly token spend exceeds $3K, evaluate Option C as a cost lever; when spend exceeds $15K and MLOps capacity exists, evaluate Option B. Map your top three AI workloads against the framework in this article, identify which option each should be on, and define the migration path.
The governance layer — model approval, compliance mapping, audit logging — sits on top of all of this. For that layer in detail, see compliance and governance considerations for open-weight AI.
For a broader view of open-source AI model options and where this framework fits in the full open-weight landscape, the pillar article has the context.
Frequently Asked Questions
Is self-hosting an open-weight model actually cheaper than using a proprietary API?
It depends on token volume and whether MLOps staffing is in the TCO calculation. At low volume (under 10M tokens/month), self-hosting is almost always more expensive once staffing is included — managing a cluster costs $200K–$500K+ per specialist annually. At medium volume (10M–50M tokens/month), managed open-weight beats proprietary API on cost without the self-hosting complexity. At high volume with existing MLOps capacity, an 8-GPU H100 cluster at $250,000–$300,000 eliminates per-token fees — work out whether your current API spend justifies the capital. The most common mistake is comparing hardware cost against API spend while ignoring staffing.
What team size do I need to self-host an open-weight model?
There’s no fixed minimum, but you need at least one dedicated MLOps or ML platform engineer as a prerequisite. A team of 8–10 engineers can support self-hosting if one engineer is dedicated to ML platform work. Smaller teams should use Option C. The gate isn’t headcount — it’s specialisation. CUDA expertise, inference stack management, and on-call capacity are the actual requirements.
Can AWS Bedrock replace a proprietary API for coding agent workloads?
Yes, for most coding agent workloads. Qwen3 Coder Next achieves competitive SWE-Bench Pro scores, matching or outperforming models with far larger active compute. AWS Bedrock provides access to Qwen3 Coder Next, DeepSeek V3.2, and other top coding models via a managed API with OpenAI compatibility. Evaluate on your own codebase — model performance varies significantly by language and framework. For model-by-model detail, see the coding agent model comparison and the Bedrock managed open-weight guide.
What is vendor lock-in and why does it matter for AI APIs?
Vendor lock-in occurs when source code and critical data become tied to a vendor’s environment, making switching difficult. In the AI context, lock-in builds through custom prompt engineering, feature dependencies, pricing commitments, and the engineering cost of re-testing migrated workloads. Open-weight models mitigate this because the weights are portable — move the same model to a different provider without retraining.
What is a managed open-weight service and how is it different from a proprietary API?
A managed open-weight service hosts publicly available open-weight model weights — Llama, Qwen, Mistral, DeepSeek — on cloud infrastructure behind a managed API. The user experience is similar to a proprietary API, but the model isn’t proprietary. If the provider changes pricing or terms, you can migrate the same model elsewhere or self-host. The difference from self-hosting: no GPU infrastructure, CUDA, or model deployment management — the cloud provider handles all of that. Examples: AWS Bedrock, Azure AI Foundry, Groq, Baseten.
What is total cost of ownership (TCO) for an AI deployment?
TCO is the full financial cost of a deployment option — direct costs (API spend or hardware) plus all indirect costs (engineering time, MLOps staffing, on-call burden, CUDA maintenance, staff turnover risk). For proprietary API: token spend plus integration time. For self-hosted: GPU hardware amortisation plus MLOps staffing plus on-call overhead. For managed open-weight: token spend plus integration time. TCO comparisons that exclude staffing cost systematically understate the true cost of self-hosting.
What is LLM sovereignty and why does it matter?
LLM sovereignty is the degree of organisational control over model weights, inference infrastructure, and data flows. The five-class sovereignty spectrum runs from Class 1 (browser-based AI tools — zero control) to Class 5 (self-hosted GPU cluster with no external dependencies). Regulatory requirements in HealthTech and FinTech may require data to never leave a controlled environment. Compliance requirements act as a hard filter: HIPAA PHI data requires at minimum a BAA-covered managed service; GDPR personal data requires EU data residency.
When should I start evaluating the move from a proprietary API to an open-weight option?
When monthly API spend reaches $3K–$5K/month, when compliance requirements make proprietary API untenable, or when vendor lock-in risk becomes a board-level concern. Start with a single workload migration — RAG pipelines or batch inference are the lowest-risk starting points — not a full platform switch. Run the new option in parallel for 4–8 weeks before full cutover. The trigger for evaluating Option B is token spend exceeding $15K/month with existing MLOps capacity.
How do compliance requirements affect the build-vs-buy decision?
They act as a hard filter before cost analysis begins. HIPAA: PHI data cannot transit a proprietary API without a signed BAA — not all providers offer them. GDPR: personal data processing must occur within the EU or under an adequacy decision — this eliminates US-only managed services for EU customer data. SOC 2: audit trail requirements vary by tier; self-hosting provides the most flexibility. For full governance treatment, see compliance and governance for open-weight AI.
What is the difference between open-weight and open-source AI models?
Open-weight models release the trained weights for download and deployment but may not release training code, data, or full architectural details. Open-source implies access to the full software stack. Most enterprise-relevant models — Llama, Qwen, DeepSeek, Mistral — are open-weight, not fully open-source. The distinction matters for academic reproducibility, not for commercial deployment. Apache 2.0 licensing on most leading open-weight models permits unrestricted commercial use, including fine-tuning and redistribution. “Open-weight” is the preferred term in enterprise contexts.
