context rot
The fall in a model's accuracy as its input grows longer, even on simple tasks and well within the context window.
In more detail
Chroma's 2025 study measured it across many models, and found that distracting but similar passages make it worse. Keep contexts short and relevant rather than filling the window because it is there.
Related terms
- context distraction — A long context pulling a model towards repeating its history or attending to similar but irrelevant passages instead of the task.
- lost in the middle — The finding that models use information at the start or end of a long context better than information in the middle.
- context engineering — Choosing and arranging everything a model reads on each call, from instructions and tools to history, memory and retrieved data, within a token budget.
Taught in
- Tokens And The Context Window — AI Engineering: LLM Apps, RAG & Agents
- When Context Fails — AI Engineering: LLM Apps, RAG & Agents
References
- Chroma: Context Rot: How Increasing Input Tokens Impacts LLM Performance (2025-07-14)
- Drew Breunig: How Long Contexts Fail (2025-06-22)