Never Bet on One Model

A practical framework for building AI systems that stay operational when your primary model fails, gets expensive, or disappears overnight.

Most organisations have found a model that works. They shipped it, and they are hoping it stays stable. That was a reasonable bet two years ago. It is not a reasonable bet now. Vendor concentration in AI introduces two failure modes that are no longer hypothetical: sudden availability loss driven by regulatory intervention or security incidents, and economic volatility as providers move from flat licensing to consumption based token pricing that can multiply costs tenfold overnight.

The answer is not to find a better single model. It is to stop treating any individual model as a foundation and start treating your model portfolio the way you treat any other critical infrastructure: with redundancy, documented fallbacks, and a regular maintenance schedule.

Start with two. Aim for three.

The minimum viable setup is two models. One primary, one fallback. But two is brittle in a different way: if your primary goes down and your fallback has a quality problem or a pricing spike, you have no buffer. Three gives you real optionality: one model for cost sensitive, high volume tasks; one for quality critical reasoning; and one as a genuine emergency backup. That said, three models maintained loosely is worse than two models maintained rigorously. If your team is small or your AI footprint is focused, commit to two and do the work properly rather than spreading attention across three.

The trend data supports this direction. According to Gartner’s Market Guide for AI Gateways, only 25% of software engineering teams building multi model applications were using AI gateways in 2025. Gartner predicts that figure reaches 70% by 2028, a signal that the industry is moving toward multi model infrastructure as a default, not an exception.

Use benchmarks to find capability peers, not a winner.

Most teams look at benchmark leaderboards to find the best model. That is the wrong question for a resilience strategy. You are not looking for a winner. You are looking for models that score closely enough on the tasks that actually matter to you that they are interchangeable from a quality standpoint.

This means running benchmarks that are specific to your use case rather than general rankings. If your core use case is document summarisation, run SCROLLS or QMSum. If it is structured data extraction, build your own eval set from real production examples. If it is multi step reasoning over internal knowledge, run something like HotpotQA adapted to your domain.

The goal is a narrow, honest answer to one question: can model B produce outputs my users would accept if model A suddenly vanished? Define your own acceptable threshold before you start testing, based on what your users actually notice. There is no universal number. Set the bar in advance so you are not rationalising results after the fact.

You are not looking for the best model. You are looking for models that can cover for each other.

Evaluate each model independently against your NFRs.

After you have confirmed capability parity, evaluate each model on its own against your non functional requirements. Do this independently, not comparatively. The moment you frame it as a head to head, you stop noticing absolute problems and start rationalising relative ones.

Forrester’s State of Enterprise Risk Management, 2025, found that 36% of multinationals flagged overreliance on technology as a major risk driver. Firms without board level risk visibility were 20% more likely to suffer six or more critical events. These numbers are not about AI specifically, but the pattern maps directly: concentration without contingency is the consistent failure mode.

The four dimensions that matter most are these. Latency: what is the p95 response time under your expected load, and is it within your user experience threshold? Reliability: what is the actual uptime and rate of degraded responses over a thirty day window, not the SLA on paper? Cost: what does total inference cost look like at your current token volume, and what does it look like if you scale by a factor of three? Consistency: if you run the same prompt one hundred times, how much variance do you see in output structure and quality? That last one matters enormously for automated pipelines.

Document these numbers per model, per month. This is your baseline for the review cycle described at the end.

Test the same prompts independently. Expect divergence.

Once you have selected your models, run your actual production prompts against each of them independently. Do not show the outputs to the same person at the same time. Have someone evaluate model A outputs one day and model B outputs the next, blind. You want to know whether the quality is genuinely equivalent or whether you have been unconsciously grading on a curve.

What you will almost certainly discover is that some prompts perform well across all your models and some do not. Instruction following conventions differ between models. Claude tends to be literal and thorough. GPT family models often compress and interpret. Gemini handles very long contexts differently to both. A prompt written for one will sometimes degrade noticeably on another.

The honest answer is that it depends on how different your models are. A single universal prompt can easily become the lowest common denominator across all three rather than genuinely good on any. If your blind evaluation reveals meaningful quality gaps, maintain separate prompt sets from the start. The maintenance overhead is real, but so is the cost of mediocre outputs at scale. Whatever you choose, version control it and review it as part of your regular cycle.

Train your humans as the backup of last resort.

Models are not the only fallback. For workflows where AI has replaced or augmented a human decision, you need a documented process for a human to step back in when every model option fails or is compromised. This is not a hypothetical. Regulators are increasingly requiring it. And operationally, there will be moments, particularly during security incidents or novel failure modes, where you do not want to trust any automated system.

Deloitte’s State of AI in the Enterprise 2026 found that only one in five companies has a mature governance model for autonomous AI agents. That gap matters here: if you have not defined where humans must stay in control, you have not actually built a backup. You have just assumed one exists.

The human backup needs three things to function. First, they need to know the process they are covering, not in the abstract, but step by step, including what the AI was doing, what inputs it was receiving, and what outputs it was producing. Second, they need a clear trigger: what specific condition activates them, and who makes that call? Third, and this is the one most organisations skip, they need to know when to stop. A human acting as AI backup under pressure will start to drift: overexplaining, adding caveats, slowing down. Define the exact signal that tells them the AI system is back online and their role is done.

Run a dry run of this handoff at least once a quarter. Not a tabletop exercise, but an actual handoff where a real person does the work for a defined window and you measure quality and speed. You will learn things about your process that no checklist will surface.

Your humans are not a fallback of convenience. They are a fallback of last resort, and they need to be ready for it.

Account for geography, not just capability.

One complication that is easy to overlook: the geopolitical layer is shrinking your options even as the model market is growing. Gartner published a prediction in January 2026 that by 2027, 35% of countries will be locked into region specific AI platforms using proprietary contextual data. The driver is sovereignty, not performance. Governments and regulators are increasingly requiring that AI processing happens domestically, on infrastructure aligned with local laws. A model that is technically the best fit for your use case may simply not be legally available in all of your operating jurisdictions.

Deloitte has pointed out a related concentration risk: 90% of all AI compute today is managed by US and Chinese companies. For any organisation operating across multiple regions, that is a structural constraint on your fallback options. Their direct advice to enterprise leaders: invest in interoperability as protection against lock in, and design for portability so the flexibility to switch is itself a form of resilience.

Review the whole setup every two months.

The model landscape moves fast enough that a portfolio decision you made in March may be wrong by May. New models are released. Pricing structures change. Capability gaps that once seemed fixed close or widen. Sovereignty rules shift. Two months is the right cadence: long enough to accumulate meaningful usage data, short enough to catch drift before it becomes a crisis.

Each review should cover the same checklist. First, pull your NFR numbers for each model from the past sixty days and compare them to your baseline and flag anything that has moved meaningfully relative to the thresholds you defined upfront. Second, rerun your benchmark eval set on any model that has had a significant version update; do not assume version N+1 is better than version N on your specific tasks. Third, run your production prompts against each model again and do a fresh blind evaluation: tastes change, your prompts change, your users change. Fourth, check the market: has a new model entered a capability tier you should be testing? Has a model in your portfolio attracted regulatory attention in a market where you operate?

Document every review in a single living document. The point is not to create bureaucracy. The point is that when something goes wrong at eleven at night, the person on call has a current, accurate picture of your model portfolio and can make decisions quickly.

The discipline is the advantage.

Most organisations are not doing any of this. They found a model that works, they shipped it, and they are hoping it stays stable. McKinsey’s State of AI in 2025 found that only one third of companies have begun to scale their AI programs at enterprise level. The majority are still experimenting, which means the majority have not yet built the infrastructure that makes AI operations survivable when something goes wrong.

The organisations that build this discipline now will be the ones that keep running when the next disruption hits. And there will be a next disruption. Multi model strategy is not an engineering luxury. It is the price of using AI in anything that matters, and the foundation on which everything else you build will depend.

All articles on this site are written by me. I use AI to assist with final formatting and editing before publication.