temperature
The sampling setting that divides the logits before softmax: values below one sharpen the distribution towards the likeliest token, values above one flatten it.
In more detail
Temperature 0 is close to greedy decoding and suits extraction and classification; higher values give more varied wording. Many reasoning models accept only their default, and Anthropic advises changing temperature or top p, not both.
Example
"temperature": 0.2Related terms
- sampling (decoding) — Choosing each next token at random from the model's probability distribution, shaped by settings such as temperature, top k and top p.
- logits — The raw, unnormalised scores a model assigns to every token in its vocabulary as candidates for the next position.
- top p sampling — A sampling rule that keeps the smallest set of likeliest tokens whose probabilities add up to p, and samples among them.
Taught in
- How The Next Token Is Chosen — AI Engineering: LLM Apps, RAG & Agents
- Choosing A Model — AI Engineering: LLM Apps, RAG & Agents
- The Request And The Response — AI Engineering: LLM Apps, RAG & Agents
- Anatomy Of A Prompt — AI Engineering: LLM Apps, RAG & Agents
- Zero-Shot And Few-Shot — AI Engineering: LLM Apps, RAG & Agents
- Reasoning Techniques — AI Engineering: LLM Apps, RAG & Agents
- Structured Output — AI Engineering: LLM Apps, RAG & Agents
- From Prompts To Context — AI Engineering: LLM Apps, RAG & Agents
- Memory And Compaction — AI Engineering: LLM Apps, RAG & Agents
- Workflows Before Agents — AI Engineering: LLM Apps, RAG & Agents
- Threat-Model An LLM App — AI Engineering: LLM Apps, RAG & Agents
- Prompt Injection — AI Engineering: LLM Apps, RAG & Agents
- Excessive Agency — AI Engineering: LLM Apps, RAG & Agents
- Model-Graded Evals — AI Engineering: LLM Apps, RAG & Agents
- Regression Tests In CI — AI Engineering: LLM Apps, RAG & Agents
References
- Anthropic: Messages API reference (temperature, top_k, top_p, stop_sequences, stop_reason)
- OpenAI: Reasoning models guide (unsupported sampling parameters)