How to make your AI app survive when a model suddenly disappears

Every developer building on AI eventually learns the same lesson the hard way: the model you depend on is not guaranteed to be there. It can be rate-limited at peak traffic, return a refusal, hit an overload error, or — as happened to Claude Fable 5 in June 2026 — be switched off entirely for reasons that have nothing to do with your code. The apps that survive these moments aren't the ones with the best prompt. They're the ones that planned for the model being gone.

This guide walks through a practical, layered approach to resilience that works whether you call one provider directly or route through a gateway.

Step 1: Never hardcode a single model

The most common fragility is a model name baked directly into a request. The moment that string stops working, every call fails. Instead, define your models as configuration — a small priority list, not a constant:

MODEL_CHAIN = [
    "claude-opus-4-8",      # primary
    "claude-sonnet-4-6",    # same provider, cheaper/faster
    "gpt-5.5",              # different provider entirely
]

This one change means switching providers during an incident is a config edit, not a code deploy. Notice the third entry is a different vendor. Same-provider fallbacks protect you from a single model going down; cross-provider fallbacks protect you from the whole provider going down — or being ordered offline.

Step 2: Use a fallback chain, not a single try

The pattern is simple: try each model in order, move to the next on failure, and only surface an error if the whole chain is exhausted.

def call_with_fallback(messages):
    last_error = None
    for model in MODEL_CHAIN:
        try:
            return client.complete(model=model, messages=messages)
        except (RateLimitError, OverloadError, RefusalError) as e:
            last_error = e
            continue  # try the next model
    raise AllModelsFailedError(last_error)

What you fall back on matters. Retrying the same model on a transient overloaded (HTTP 529) error makes sense after a short backoff. But an authentication error or a malformed request will fail identically on every model — those should surface immediately rather than burning through your whole chain.

Step 3: Lean on the platform when it offers fallbacks natively

You don't always have to hand-roll this. Several platforms now do it server-side:

• Anthropic's API added a beta fallbacks parameter that retries a refused request on an alternate model in a single round trip — Fable 5, for example, was configured to fall back to Opus 4.8 when its classifiers declined a request. Billing follows the model that actually served the response.
• OpenRouter lets you pass a models array in priority order; it automatically fails over on context-length errors, moderation flags, rate limits, and downtime, billing you at the rate of whichever model answered.
• LiteLLM ships a Router with built-in retries, fallbacks, and load balancing across deployments and vendors, which you self-host for maximum control.

A gateway like OpenRouter or LiteLLM gives you one unified endpoint across many providers, so a cross-provider fallback is just another entry in a list rather than a second SDK to integrate.

Step 4: Add a circuit breaker so you fail fast

If a model is clearly down, you don't want every request waiting for it to time out before falling through. A circuit breaker tracks recent failures and, once a threshold is crossed, skips the broken model entirely for a cooldown period. Production teams in 2026 commonly land around five consecutive failures to trip the breaker and a ~60-second cooldown before testing recovery. While the breaker is open, traffic goes straight to the next model in the chain — no wasted latency.

Step 5: Decide what "degraded" means for your product

Falling back to a cheaper model is not free — quality drops, and your prompts may behave differently. Plan for it:

• Test your prompts against every model in the chain. A prompt tuned for one model can produce noticeably worse output on its fallback.
• Tell the user when you've degraded if it matters (e.g. "running in fast mode"), rather than silently shipping lower-quality results.
• Log which model actually served each request so you can see, after an incident, how much traffic the fallbacks absorbed.

The takeaway

Resilience isn't a feature you bolt on after launch — it's a small amount of structure you add early: configuration instead of constants, a priority chain instead of a single call, a circuit breaker to fail fast, and tested prompts at every level. The teams whose apps stayed up during June's Fable 5 shutoff weren't lucky. They'd simply already answered the question every AI builder should ask on day one: what happens when this model isn't there tomorrow?