Every team building with AI eventually hits the same fork in the road: do you call a closed model through an API, or do you run an open-weight model you can host yourself? The marketing around this choice makes it sound like a values question. It is not. It is an engineering and business decision about who holds the model weights, who controls the cost curve, and who absorbs the risk when something breaks.
This guide gives you the full picture without the tribalism. We will define what "open" and "closed" actually mean, since the terms are slippery, then work through capability, cost, control, compliance, and the operational reality of running models in production. By the end you should be able to look at a specific use case and say with confidence which side of the line it belongs on.
What Open and Closed Source Actually Mean
The labels hide more than they reveal. "Closed source" models like the GPT and Claude families are served exclusively through an API. You send tokens, you get tokens back, and you never touch the weights. The provider controls versioning, safety filtering, rate limits, and pricing.
"Open source" is the messier term. Most models people call open are really open-weight: the trained parameters are downloadable, but the training data and full training code often are not. Llama, Mistral, Qwen, and Gemma fall here. A smaller set, like some OLMo releases, are genuinely open across data and code. The distinction matters because an open-weight license can still restrict commercial use, redistribution, or scale.
The Three Axes of Openness
- Weights: Can you download and run the parameters yourself?
- License: Does the license permit your specific commercial use and scale?
- Reproducibility: Are the data and training recipe public enough to rebuild or audit?
Most decisions hinge on the first two. Reproducibility matters mainly for research and high-assurance regulated work.
Capability: The Gap Is Narrowing but Not Gone
For years the assumption was that closed frontier models won every benchmark and open models trailed by a generation. That gap has compressed sharply. Top open-weight models now match closed models on many reasoning, coding, and summarization tasks, especially at mid-range sizes.
Where closed models still tend to lead is the absolute frontier: the hardest multi-step reasoning, long-context coherence, and the newest multimodal capabilities. If your work depends on the single best model available this quarter, closed is usually where it lives first. If your work needs "very good and consistent," open-weight models cover an enormous amount of ground.
For a deeper walk through specific deployments, see our real-world examples and use cases.
Cost: API Convenience Versus Infrastructure Ownership
Closed models charge per token. There is no upfront cost, no GPUs to provision, and you scale to zero when idle. This is unbeatable for low or spiky volume. The catch is that per-token pricing becomes brutal at high, steady volume, and you have no leverage to negotiate the unit economics down except by switching providers.
Open-weight models flip the equation. You pay for GPUs—rented or owned—plus the engineering time to serve, monitor, and update them. At low volume this is wildly inefficient. At high, predictable volume it can be dramatically cheaper per token, and the cost is fixed rather than variable.
A Rough Decision Heuristic
- Under ~1M tokens/day, bursty: Closed API almost always wins on total cost.
- Tens of millions of tokens/day, steady: Self-hosted open-weight often wins, sometimes by 5-10x per token.
- In between: Model it both ways; the answer depends on your latency and uptime requirements.
Control, Privacy, and Data Residency
This is where open-weight models have a structural advantage that no amount of closed-model engineering can fully match. When you host the weights, data never leaves your environment. For healthcare, defense, finance, and anyone bound by strict data residency rules, that property is sometimes non-negotiable.
Closed providers have responded with enterprise tiers, zero-retention API options, and regional hosting. These are real and often sufficient. But they are contractual guarantees, not architectural ones. You are trusting a third party rather than controlling the boundary yourself.
Customization Depth
Closed models let you fine-tune within the limits the provider exposes. Open-weight models let you do anything: full fine-tuning, quantization, LoRA adapters, distillation into smaller models, and surgical modification of behavior. If your edge depends on a model that behaves differently from what competitors can rent, open is the path.
Operational Reality and Hidden Costs
The brochure comparison ignores the part that sinks teams: running an open-weight model in production is an ongoing engineering commitment. You own inference optimization, GPU availability, autoscaling, observability, security patching, and the painful work of evaluating and migrating to new model versions yourself.
Closed providers absorb all of that. You trade control for the fact that a team of specialists keeps the model fast, available, and current. Many organizations underestimate this and discover that their "cheaper" self-hosted setup costs more once they price in the senior engineers babysitting it. The common mistakes article covers this trap in detail in our 7 common mistakes guide.
Putting It Together: A Portfolio Approach
The mature answer is rarely all-or-nothing. Sophisticated teams run a portfolio: a closed frontier model for the hardest tasks and prototyping, and open-weight models for high-volume, latency-sensitive, or privacy-bound workloads. Routing logic sends each request to the cheapest model that can do the job.
This hybrid stance hedges vendor risk, controls cost, and keeps you close to the frontier. To structure that decision repeatably, our framework article gives you a named model you can reuse.
Vendor Risk and Strategic Independence
Beyond cost and capability, there is a strategic dimension that quietly shapes many decisions: dependence. Building a business entirely on one closed provider means your product's economics, capabilities, and availability are subject to another company's roadmap, pricing, and policy changes. A price increase, a deprecated model, or a usage policy shift can ripple straight into your product overnight.
Open-weight models are the natural hedge. Even if you never self-host today, the existence of capable open models that you could run gives you negotiating leverage and an exit path. Many teams keep an open model warm in their stack precisely as insurance, not because they use it heavily. The optionality has value even when unexercised.
This does not mean closed providers are dangerous; they are excellent and most teams should use them. It means that betting your entire business on a single external dependency, of any kind, deserves a deliberate look. A modest investment in portability—the thin model interface, a tested open-model fallback—buys real strategic resilience for very little ongoing cost.
Frequently Asked Questions
Are open source AI models actually free?
The weights are free to download under permissive licenses, but running them is not free. You pay for GPU compute, engineering time, and operational overhead. For most workloads the dominant cost is infrastructure and people, not licensing.
Can closed models match open models on privacy?
They can get close through zero-retention APIs and regional hosting, which satisfy many compliance regimes. But these are contractual promises rather than architectural guarantees. If your requirement is that data physically never leaves your environment, only self-hosted open-weight models deliver that.
Which is better for a startup just getting started?
Almost always a closed API. The speed to a working product, zero infrastructure burden, and access to frontier capability outweigh cost concerns at low early volume. Revisit the decision once usage grows large and predictable.
Do open-weight licenses really restrict commercial use?
Some do. Certain licenses cap usage at a scale threshold or restrict specific applications. Always read the actual license for the specific model version before building a business on it, because terms vary between models and even between releases of the same model family.
How often do I need to migrate models?
With closed APIs, the provider may deprecate versions on their schedule, forcing periodic migration. With self-hosted open models, you migrate on your own timeline but own all the evaluation work. Either way, budget for re-testing your prompts and pipelines a few times a year.
Key Takeaways
- "Open" usually means open-weight, not fully open data and code; read the license for your specific use and scale.
- Capability gaps have narrowed; open models cover most needs, while closed models still lead at the absolute frontier.
- Closed APIs win on cost at low or bursty volume; self-hosted open models win at high, steady volume.
- Self-hosting gives architectural privacy and unlimited customization but adds a real, ongoing engineering burden.
- The strongest strategy is usually a hybrid portfolio with routing, not a dogmatic commitment to one side.