just-in-time context

Keeping bulky data out of the prompt and giving the model references, such as file paths or ids, that it loads with a tool only when needed.

In more detail

Agents such as coding assistants work this way: they list and search before reading. It keeps the context small, at the price of extra tool calls and some latency.

Related terms

Taught in

References

Certification courses