model fallback
Sending a request to a second model or provider when the first keeps failing, so the user still gets an answer.
In more detail
A fallback model needs the same evaluation as the main one, because its answers may differ in quality, format and cost. Decide in advance which errors trigger it and log when it is used.
Related terms
- retryable error — An error that may succeed if tried again, such as a connection reset or a busy response.
- model router — A service that sends each request to one of many models or providers behind a single API, often with automatic fallbacks.
Taught in
- What An AI Engineer Does — AI Engineering: LLM Apps, RAG & Agents
- Tokens And The Context Window — AI Engineering: LLM Apps, RAG & Agents
- Choosing A Model — AI Engineering: LLM Apps, RAG & Agents
- When The Call Fails — AI Engineering: LLM Apps, RAG & Agents
- Anatomy Of A Prompt — AI Engineering: LLM Apps, RAG & Agents
- Structured Output — AI Engineering: LLM Apps, RAG & Agents
References
- OpenRouter: Provider selection (order, fallbacks, data collection, ZDR)
- AWS Architecture Blog: Exponential Backoff And Jitter