softmax
The function that turns a list of scores into probabilities by exponentiating each one and dividing by the total.
In more detail
Applied to a model's logits, softmax gives the probability of every possible next token. Large scores dominate after exponentiation, which is why dividing the logits by a temperature first changes how peaked the distribution is.
Related terms
- logits — The raw, unnormalised scores a model assigns to every token in its vocabulary as candidates for the next position.
- temperature — The sampling setting that divides the logits before softmax: values below one sharpen the distribution towards the likeliest token, values above one flatten it.
Taught in
- How The Next Token Is Chosen — AI Engineering: LLM Apps, RAG & Agents
References
- Holtzman et al. (2019): The Curious Case of Neural Text Degeneration (nucleus sampling)
- Google Machine Learning Crash Course: Softmax