max tokens
The request parameter that caps how many tokens the model may generate in one reply.
In more detail
When the cap is reached the reply stops mid-sentence and the stop reason says max_tokens (Anthropic) or length (OpenAI), so a truncated reply can look complete unless you check. Set it high enough for the longest valid answer, and treat a capped reply as a failure for structured output.
Example
"max_tokens": 1024Related terms
- stop reason — The field in a model reply that says why generation ended, such as a natural end, the token cap, a stop sequence, a tool call or a refusal.
- context window — The maximum number of tokens a model can take into account at once, covering the prompt and the reply together.
Taught in
- Tokens And The Context Window — AI Engineering: LLM Apps, RAG & Agents
- How The Next Token Is Chosen — AI Engineering: LLM Apps, RAG & Agents
- The Request And The Response — AI Engineering: LLM Apps, RAG & Agents
- Zero-Shot And Few-Shot — AI Engineering: LLM Apps, RAG & Agents
- Structured Output — AI Engineering: LLM Apps, RAG & Agents
- When Context Fails — AI Engineering: LLM Apps, RAG & Agents
- Function Calling — AI Engineering: LLM Apps, RAG & Agents
- The Agent Loop — AI Engineering: LLM Apps, RAG & Agents
References
- Anthropic: Messages API reference (temperature, top_k, top_p, stop_sequences, stop_reason)
- OpenAI: Chat Completions API reference (temperature, penalties, logprobs, stop)