user feedback
Ratings or comments from users on model answers, recorded and linked to the call that produced them.
In more detail
The GenAI conventions model it as an evaluation result event linked by response id. Read the traces behind poor ratings and turn real failures into golden-set cases.
Related terms
- GenAI span — A trace span for one model call, named after the operation and model, such as "chat claude-sonnet", carrying the GenAI convention attributes.
- golden set — A curated set of inputs with known good outcomes, used to check that an AI feature still behaves after every change.
Taught in
- The Agent Loop — AI Engineering: LLM Apps, RAG & Agents
- Workflows Before Agents — AI Engineering: LLM Apps, RAG & Agents
- Regression Tests In CI — AI Engineering: LLM Apps, RAG & Agents
- Tracing And Monitoring — AI Engineering: LLM Apps, RAG & Agents
- Capstone: The Tasks Assistant — AI Engineering: LLM Apps, RAG & Agents
References
- OpenTelemetry GenAI conventions: Events (gen_ai.evaluation.result)
- Anthropic: Demystifying evals for AI agents