just-in-time context
Keeping bulky data out of the prompt and giving the model references, such as file paths or ids, that it loads with a tool only when needed.
In more detail
Agents such as coding assistants work this way: they list and search before reading. It keeps the context small, at the price of extra tool calls and some latency.
Related terms
- context engineering — Choosing and arranging everything a model reads on each call, from instructions and tools to history, memory and retrieved data, within a token budget.
- tool call — A model's structured request to run one tool with given arguments, returned instead of, or alongside, text.
Taught in
- From Prompts To Context — AI Engineering: LLM Apps, RAG & Agents
- How RAG Works And When To Use It — AI Engineering: LLM Apps, RAG & Agents
- Planning, Memory And Many Agents — AI Engineering: LLM Apps, RAG & Agents