Open source LLMs run 85-90% cheaper per token. So why does closed source still own ~90% of the enterprise AI market?
The cost math was never the whole story
If price alone decided the market, this wouldn't even be a debate. Cheaper tokens only matter if you already have the GPUs, the in-house LLM expertise and the guardrails to run them safely.
Why most organizations still choose closed source
- In-house expertise. Few teams know how to fine-tune and optimize open weights.
- Guardrails. Safety and compliance ship built into closed APIs, not with raw weights.
- GPU capex. Buying and racking GPUs isn't realistic for most IT organizations.
- Quality. Historically, open models couldn't match closed-model quality.
Two ways to deploy open models
Dedicated GPU infrastructure gives full control and dedicated capacity, the right fit for consistent, large-scale workloads. Cloud-hosted instances rent GPU capacity through the hyperscalers: fast to deploy and elastic to scale. The open models are the same either way. The difference is who owns the GPUs underneath.
The quality gap is closing fast
In the LLaMA era, open models were usable for narrow tasks but fell short on nuanced, high-stakes reasoning. GLM-5.2 and others are now closing the gap for more than 90% of enterprise workloads. The question is changing from “can we use open source?” to “why aren't we routing more traffic there?”
Privacy isn't the real difference
A closed model's API and an open model on the same hyperscaler cloud offer the same protections: zero data retention, no training on your data, the same compliance boundary and the same cloud region. The real difference isn't privacy. It's the token cost.
One harness for every model
Our healthcare work began on open source models running on dedicated GPU infrastructure, then moved to cloud-hosted instances, and eventually to routing across both open and closed models through our own harness as the cost and quality tradeoffs shifted. That trajectory is why "open vs. closed" has never been the right framing: the real constraint has consistently been in-house expertise, guardrails, and infrastructure flexibility, not the model itself.
That harness is Dev Studio on InferAgents.ai: routing, guardrails and security in one layer between open and closed models and your agents, memory and app logic. One layer. Every model. No dependency.
It's also why we're now looking well beyond healthcare. GLM-5.2 and others are closing the quality gap quickly, and the same PII, audit, and zero-tolerance data requirements that shaped our approach in healthcare are surfacing just as sharply in finance and telecom. A harness built for one regulated industry generalizes cleanly to any of them.
