When an AI route fails live: what must a fallback preserve?

An AI workflow can become available again without being ready to continue. A substitute model may respond promptly yet lack approval for the information, fail the task’s quality checks or repeat an action that already happened.

During a live failure, the useful question is not simply “What else can answer?” It is: “Can this work continue through an already-authorised route, without changing its obligations or duplicating an action?” Sometimes the correct answer is to hold.

This guide covers unexpected failure and temporary fallback within an approved workflow. It does not authorise new candidates, plan a model retirement or clear a wider provider incident.

Start here

• Locate the failure: distinguish a failed AI request from an uncertain downstream action.
• Check the substitute: confirm that existing approval still covers this exact task, information and route.
• Close the task deliberately: accept only a checked result; otherwise record a hold, escalation or approved manual outcome.

First, establish what already happened

Before trying another route, identify the task, the failed attempt and its last known state. Did the AI request return anything? Did the workflow request a write, send a message or start another external action? What acknowledgement was received?

A timeout does not prove that nothing happened. In particular, a missing acknowledgement from a case-system write leaves a different problem from a failed generation request. The destination may have changed even though the caller cannot see confirmation.

Use the task ID, request key and authorised destination readback to reconcile that uncertainty. Do not regenerate and replay the whole workflow merely because its final response is missing. A new model cannot resolve whether an earlier write succeeded.

AWS’s idempotent-API design discussion illustrates why a lost response can require reconciliation and why the service must enforce duplicate-request protection. It does not verify the destination used by your workflow.

A request key helps connect records. Its presence alone does not prove idempotent execution—the property that repeating an operation will not duplicate its effect. That protection must be established in the components performing the action.

If the relevant state cannot be checked, retain the uncertainty and hold the action rather than recording “not written”.

Run a compact fallback preflight

Use the workflow’s existing approval record, not a fresh shortlist assembled under pressure. This is a check that approval still applies, not a substitute for candidate evaluation.

Check	Evidence needed now	Hold if…
Identity	Exact eligible model and provider route; a way to identify what actually runs	The candidate is unnamed, merely similar or outside the approved set
Information and access	Approval for this data class, destination, account and task scope	Continuation widens access or introduces an undocumented data path
Acceptance	Relevant quality test, required output checks and available reviewer	The substitute cannot meet the existing review standard
Limits	Remaining cost, attempt and elapsed-time allowances	The next attempt exceeds a bound or its cost cannot be assessed
External effects	Known action state and a reconciliation method	A previous write or dispatch may already have happened

Changing providers may change where information goes; changing models may change the result. Neither change becomes acceptable solely because the request can be routed.

Also establish which component owns the next attempt. A router retry and an application replay are not the same operation. Independent retries can multiply attempts and cost, or repeat downstream actions. AWS’s timeouts and retries discussion describes retry amplification across service layers; check the retry behaviour of every component in your own workflow.

No eligible candidate is a valid stopping state. It is not an invitation to broaden the provider list, permissions or permitted information.

Classify the failure before choosing an action

The failure category determines what needs checking. These are decision rules, not instructions to enable automatic retries.

Condition	Immediate response
Timeout before output	Inspect the attempt and downstream state. Consider an eligible substitute only when uncertainty about external effects is resolved.
Lost write acknowledgement	Hold replay. Reconcile the task against the destination before deciding what remains to do.
Moderation refusal	Hold for policy review; do not automatically seek another model that will answer.
Rate limit or service error	Consider an approved substitute within limits, then check its output before acceptance.
No eligible route remains	Stop automatic dispatch, preserve pending work and escalate to the responsible owner.

A context-validation error needs inspection too. A substitute may not accept the same input. Quietly truncating documents or removing instructions to make the request fit can change the task. Record any proposed change and obtain the applicable approval instead.

What product documentation establishes—and what it does not

The OpenRouter Model Fallbacks documentation describes an ordered model list that can try another model after an error. Documented default triggers include context validation, moderation, rate limits and downtime; the final response names the model used.

That explains a routing mechanism. It does not establish that every substitute is authorised or that every returned answer should be accepted. The recommendation to hold a moderation refusal for review is a governance judgement, not a provider guarantee.

Keep fallback destinations within the authorised set. OpenRouter’s Provider Selection documentation documents an only provider list for each request and an allow_fallbacks control for backup providers when the primary is unavailable. The current documentation also describes account-wide allowed providers as a ceiling that the request’s only list narrows. These descriptions do not verify the settings of an actual account.

A retention filter is a routing condition, not an independently verified legal or privacy outcome. These descriptions are product-specific; they are not request syntax for Sanctum Router or another system.

Five cases from a fictional tabletop

Consider a fictional professional-services firm summarising synthetic documents. Route A is primary. Route B is separately approved for the same synthetic data class, task and review standard. Route C is cheaper but has no approved data-handling record.

A human must approve the final summary before it is written to a case system. Task IDs and request keys support reconciliation. These are exercise assumptions, not observed production features.

1. A times out before producing output

No side effect is confirmed. The operator checks the attempt record and action state rather than treating the timeout as proof that nothing happened.

If those checks support continuation, B may be tried only while its exact identity, permissions, data path and cost ceiling remain approved. Retain A’s failure, the eligibility check, B’s identity, trace and review outcome. Otherwise, hold.

2. The response arrives, but the write acknowledgement is lost

The operator holds the write and reconciles the task ID against the case system. If readback confirms the accepted summary is already present, blindly repeating the write would be unnecessary and potentially harmful.

If the state remains unknown, it stays unknown. Switching to B would not resolve this failure: the unresolved question concerns the destination, not the model.

3. A produces a moderation refusal

The operator retains the refusal reason and pauses for policy review. B is not used automatically to bypass it.

Any subsequent decision must identify the policy owner, the permitted scope and the authority for an applicable exception. A technical fallback setting does not supply that authority.

4. A rate-limits; B answers but fails the quality test

B returns a summary, but it fails the prepared output-quality fixture—a test case with defined expected results. The operator holds the result or uses the approved manual process.

Record the failed check, reviewer rejection and route identity. Restored availability has not restored an acceptable deliverable, and a fluent answer does not justify weakening the standard.

5. Only C remains reachable

Automatic dispatch stops. C’s lower price and availability do not fill the missing data-handling approval.

Preserve the pending task, record why C was excluded and name the escalation owner. The owner can arrange approved manual handling, defer the work or seek a separately authorised change.

Simulation boundary: these are expected decisions in a controlled tabletop, not measured results. They do not verify actual router failure handling, a provider outage, idempotent execution or real client-data handling. Rehearse them in an authorised sandbox with approved synthetic fixtures before any production change. A successful rehearsal still does not authorise new destinations or live data.

Know when to stop

Stop when candidates are exhausted, an attempt, time or cost limit is reached, the data path is unclear, an external action remains uncertain, a refusal needs review or output fails acceptance.

Tell the responsible person what is held and what evidence or decision is needed next. Do not conceal an operational delay by reducing review or choosing an unapproved route.

If the evidence points beyond a route-level failure to a wider affected workflow, move into the organisation’s incident process. Temporary fallback is not incident clearance or a permanent migration decision.

Leave a usable recovery record

Replace “fallback worked” with a short record that another operator can follow:

Task ID / request key:
Failure time and timezone / classification:
Original route / substitute actually used:
Last acknowledgement / destination readback:
Approval and data-handling basis:
Attempt count / time and cost-limit checks:
Quality-check result / reviewer decision:
Final state: accepted / held / completed manually / escalated
Unresolved issue / owner / next review:

Link to protected evidence rather than copying sensitive inputs into general notes. Distinguish observations, decisions and unknowns. A model-used field supports reconstruction; it cannot replace destination readback, approval evidence or review.

For candidate authorisation, return to the parent routing guide. For wider incident clearance, use After an AI provider incident, what must change before your workflow resumes?. Planned retirement belongs in Replace a retiring AI model without losing control.

A useful fallback preserves the task’s information boundary, authority, external effects and acceptance standard. When it cannot, a visible hold is the correct operational result.

Sources and limits

Selected retained sections of the linked official OpenRouter model-fallback and provider-selection documentation were compared with this guide on 9 October 2026. The fictional exercise does not establish a live outage or a deployed fallback.

No production route call, outage test, account bill or real client-data test supports this article. Recheck vendor documentation and the deployed contract, catalogue, scopes and configuration before any actual route change.