greedy decoding
Always choosing the single most likely next token, with no randomness.
In more detail
Greedy decoding is repeatable on one machine but tends to loop and repeat itself in long text. Hosted APIs approximate it with temperature 0, yet small numeric differences between servers can still change the output.
Related terms
- sampling (decoding) — Choosing each next token at random from the model's probability distribution, shaped by settings such as temperature, top k and top p.
- temperature — The sampling setting that divides the logits before softmax: values below one sharpen the distribution towards the likeliest token, values above one flatten it.
Taught in
- How The Next Token Is Chosen — AI Engineering: LLM Apps, RAG & Agents
- Reasoning Techniques — AI Engineering: LLM Apps, RAG & Agents
- Vector Indexes And Databases — AI Engineering: LLM Apps, RAG & Agents