context window
The maximum number of tokens a model can take into account at once, covering the prompt and the reply together.
In more detail
Everything the model sees on a call, from the system prompt and tool definitions to the conversation and the reply it writes, must fit in the window. Reserve room for the answer, and remember that a full window costs money and dilutes attention, so long-context accuracy often falls before the limit is reached.
Related terms
- LLM token — The unit of text a language model reads and writes: a word, part of a word, a punctuation mark or a byte sequence from the tokenizer's vocabulary.
- max tokens — The request parameter that caps how many tokens the model may generate in one reply.
- context engineering — Choosing and arranging everything a model reads on each call, from instructions and tools to history, memory and retrieved data, within a token budget.
- context rot — The fall in a model's accuracy as its input grows longer, even on simple tasks and well within the context window.
Taught in
- What An AI Engineer Does — AI Engineering: LLM Apps, RAG & Agents
- Tokens And The Context Window — AI Engineering: LLM Apps, RAG & Agents
- How The Next Token Is Chosen — AI Engineering: LLM Apps, RAG & Agents
- Choosing A Model — AI Engineering: LLM Apps, RAG & Agents
- Streaming Responses — AI Engineering: LLM Apps, RAG & Agents
- Anatomy Of A Prompt — AI Engineering: LLM Apps, RAG & Agents
- Structured Output — AI Engineering: LLM Apps, RAG & Agents
- Prompts As Code — AI Engineering: LLM Apps, RAG & Agents
- From Prompts To Context — AI Engineering: LLM Apps, RAG & Agents
- Memory And Compaction — AI Engineering: LLM Apps, RAG & Agents
- When Context Fails — AI Engineering: LLM Apps, RAG & Agents
- How RAG Works And When To Use It — AI Engineering: LLM Apps, RAG & Agents
- Chunking And Ingestion — AI Engineering: LLM Apps, RAG & Agents
- Designing Tools A Model Can Use — AI Engineering: LLM Apps, RAG & Agents
- Planning, Memory And Many Agents — AI Engineering: LLM Apps, RAG & Agents
- Threat-Model An LLM App — AI Engineering: LLM Apps, RAG & Agents
- Excessive Agency — AI Engineering: LLM Apps, RAG & Agents
- Evaluating RAG And Agents — AI Engineering: LLM Apps, RAG & Agents
- Images, Audio And Speech — AI Engineering: LLM Apps, RAG & Agents
References
- Anthropic: Context windows
- Chroma: Context Rot: How Increasing Input Tokens Impacts LLM Performance (2025-07-14)