trajectory eval
Grading the path an agent took, its sequence of tool calls, in addition to the outcome it reached.
In more detail
Decide success from the final state of the system, never from what the agent says it did. Keep path checks to what matters, such as a required lookup and a budget, so a valid but different route is not punished while overreach still shows.
Related terms
- outcome grading — Judging an agent by the actual state it left behind, such as the record in the database, rather than by its own report.
- AI agent — A system in which a model decides its own next steps, calling tools in a loop until a task is done, rather than following a path your code fixed.
- AI eval — A repeatable test of an AI feature: a set of inputs, a way to grade each output, and a score you track across changes.
Taught in
- Evaluating RAG And Agents — AI Engineering: LLM Apps, RAG & Agents