context compaction
Replacing older turns of a long conversation with a model-written summary so the history fits the context again.
In more detail
Compact when the history passes a trigger, keep the newest turns word for word, and write the summariser's prompt for the agent that carries on. Test the summary for the identifiers, numbers and promises later turns depend on; Anthropic and others also offer compaction as an API feature.
Related terms
- context editing — Removing stale content, such as old tool results, from a conversation's context before the next call.
- agent memory — Information an AI application keeps outside the context window, such as user preferences or task notes, and loads back when relevant.
- context window — The maximum number of tokens a model can take into account at once, covering the prompt and the reply together.
Taught in
- From Prompts To Context — AI Engineering: LLM Apps, RAG & Agents
- Memory And Compaction — AI Engineering: LLM Apps, RAG & Agents
- When Context Fails — AI Engineering: LLM Apps, RAG & Agents
- Designing Tools A Model Can Use — AI Engineering: LLM Apps, RAG & Agents
- Planning, Memory And Many Agents — AI Engineering: LLM Apps, RAG & Agents
- Prompt Injection — AI Engineering: LLM Apps, RAG & Agents