context window

The maximum number of tokens a model can take into account at once, covering the prompt and the reply together.

In more detail

Everything the model sees on a call, from the system prompt and tool definitions to the conversation and the reply it writes, must fit in the window. Reserve room for the answer, and remember that a full window costs money and dilutes attention, so long-context accuracy often falls before the limit is reached.

Related terms

Taught in

References

Certification courses