max tokens

The request parameter that caps how many tokens the model may generate in one reply.

In more detail

When the cap is reached the reply stops mid-sentence and the stop reason says max_tokens (Anthropic) or length (OpenAI), so a truncated reply can look complete unless you check. Set it high enough for the longest valid answer, and treat a capped reply as a failure for structured output.

Example

"max_tokens": 1024

Related terms

Taught in

References

Certification courses