context distraction

A long context pulling a model towards repeating its history or attending to similar but irrelevant passages instead of the task.

In more detail

Distractors, passages that look relevant but do not answer, lower accuracy faster than plain length does. Prune and summarise rather than letting history and search results pile up.

Related terms

Taught in

References

Certification courses