outcome grading
Judging an agent by the actual state it left behind, such as the record in the database, rather than by its own report.
In more detail
Agents can claim success they did not achieve. Query the system after the run and compare it with the expected state.
Related terms
- trajectory eval — Grading the path an agent took, its sequence of tool calls, in addition to the outcome it reached.
- AI eval — A repeatable test of an AI feature: a set of inputs, a way to grade each output, and a score you track across changes.
Taught in
- Evaluating RAG And Agents — AI Engineering: LLM Apps, RAG & Agents