Four reasons single-model architectures age badly
- Task fit varies. Long-document clinical summarisation, structured extraction from scanned forms, and multi-step tool use have genuinely different leaders, and the ranking changes with each model generation.
- Cost varies by an order of magnitude. High-volume, low-complexity work — classification, routing, extraction — does not need a frontier model. Routing it to a smaller one is frequently the single largest cost lever available.
- Some data should not leave. A health system may reasonably decide certain workloads run only on internal or self-hosted models, regardless of the contractual protections around external providers.
- Providers change. Pricing, terms, deprecation schedules and availability all move. An architecture that assumes one provider converts every one of those into a project.
Model-agnostic architecture is a negotiating position as much as an engineering one. A health system that can move workloads between providers in a configuration change is a different counterparty at renewal than one that cannot.
What model-agnostic actually requires
Nearly every vendor claims to be model-agnostic. Most mean they have an adapter for more than one provider, which is the easy part. Portability is decided by everything built on top of that adapter.
| Layer | What portability requires |
|---|---|
| Prompting | Prompts held as versioned platform assets with per-model variants, not embedded in application code against one provider's quirks. |
| Tool and function calling | One internal tool definition, translated per provider. Otherwise every tool is rewritten per model. |
| Retrieval | Retrieval that is provider-independent, so the RAG layer does not have to be rebuilt when the model changes. |
| Governance and audit | One audit schema across providers, so a compliance query spans the whole estate rather than one model's logs. |
| Evaluation | A shared test set that can be run against any candidate model, which is what makes switching a measured decision rather than a leap. |
| Cost and usage accounting | Normalised token and cost reporting per user, team and agent across providers. |
The test is simple: can you change the model behind a production agent, run your evaluation set, and promote or roll back — without touching the agent's definition? If not, the architecture is multi-provider, not model-agnostic.
Routing work to the right model
Once models are interchangeable, routing becomes a live operational decision rather than an architectural commitment.
- By capability. Multi-step agentic work to models that are strong at tool use; high-volume extraction to smaller, cheaper models.
- By sensitivity. Workloads a health system prefers to keep in-house routed to internal or self-hosted models, enforced by policy rather than convention.
- By cost ceiling. Per-agent and per-team budgets that route or throttle when a threshold is reached.
- By availability. Automatic failover when a provider degrades, so an agent in a clinical workflow does not simply stop.
How this works on InferAgents
InferAgents connects to model providers only at the API level. Agents are built against the platform's own interfaces, so the model is selected per agent — or changed for an existing one — as configuration.
- OpenAI, Anthropic and Google models alongside a health system's internal or self-hosted models.
- Per-agent model selection, with a documented default and the ability to override.
- One evaluation harness that runs the same test set against any candidate model before promotion.
- Unified token, cost and usage accounting per user, team and agent across every provider.
- One audit log schema regardless of which model answered.
This is also why more than 80 agents run on one core rather than one stack per department: the expensive, reusable work sits in the platform, and the model stays a swappable component. See the InferAgents platform.
Frequently asked questions
What is multi-model AI?
Multi-model AI is an approach where an organisation runs workloads across several underlying language models rather than committing to one. Different tasks are routed to whichever model fits best on capability, cost or data-sensitivity grounds, and models can be changed as better options appear without rebuilding the applications on top.
What does model-agnostic mean in practice?
It means the underlying model is a configuration choice, not a structural dependency. Concretely: prompts, tool definitions, retrieval, governance, audit and evaluation all live in the platform rather than being written against one provider's API, so you can change the model behind a production agent, run your evaluation set and promote or roll back without editing the agent.
Should a health system use OpenAI, Anthropic or Google?
The premise is the weak part of the question. Task fit, pricing and capability all change with each model generation, so committing to one provider is the risk. The more durable decision is to build on a platform that lets you use all of them, keep certain workloads on internal models, and change your mind later at configuration cost rather than project cost.
Can we run our own internal models alongside commercial ones?
Yes, and many health systems should. A model-agnostic platform treats an internal or self-hosted model as another provider behind the same interfaces, which lets an organisation keep specific workloads in-house on policy grounds while using commercial models where they perform better.
How do we avoid AI vendor lock-in?
Keep the reusable assets — prompts, tool definitions, retrieval, evaluation sets, audit schema and governance — in a layer you control, and connect to model providers only at the API level. Then test portability rather than assuming it: change the model behind a real agent, run your evaluation set and see what breaks.