greedy decoding

Always choosing the single most likely next token, with no randomness.

In more detail

Greedy decoding is repeatable on one machine but tends to loop and repeat itself in long text. Hosted APIs approximate it with temperature 0, yet small numeric differences between servers can still change the output.

Related terms

Taught in

References

Certification courses