evaluator-optimiser
A pattern in which one model call produces an answer and another critiques it against criteria, looping until it passes or a limit is reached.
In more detail
It helps when good output is easy to recognise but hard to produce first time, as in Self-Refine. Cap the rounds, and prefer a check in code when one exists.
Related terms
- workflow (LLM) — A system in which your code fixes the sequence of model calls and tools, as opposed to an agent that chooses its own steps.
- LLM-as-judge — Using a language model to grade another model's output against a rubric, for qualities that code cannot check.
Taught in
- Workflows Before Agents — AI Engineering: LLM Apps, RAG & Agents
- Planning, Memory And Many Agents — AI Engineering: LLM Apps, RAG & Agents
References
- Anthropic: Building effective agents
- Madaan et al.: Self-Refine: Iterative Refinement with Self-Feedback