golden set
A curated set of inputs with known good outcomes, used to check that an AI feature still behaves after every change.
In more detail
Include typical cases, edge cases and every failure found in production. Mark critical cases that must always pass and give the rest a threshold with headroom, and replay recorded replies so the suite is fast and free.
Related terms
- AI eval — A repeatable test of an AI feature: a set of inputs, a way to grade each output, and a score you track across changes.
- regression — A bug where something that used to work stops working, or works worse, after a change.
- record and replay — Testing code that calls a model by recording real replies once and serving them from a file afterwards, matched by request.
Taught in
- Structured Output — AI Engineering: LLM Apps, RAG & Agents
- Prompts As Code — AI Engineering: LLM Apps, RAG & Agents
- Eval-Driven Development — AI Engineering: LLM Apps, RAG & Agents
- Regression Tests In CI — AI Engineering: LLM Apps, RAG & Agents
- Tracing And Monitoring — AI Engineering: LLM Apps, RAG & Agents
- Capstone: The Tasks Assistant — AI Engineering: LLM Apps, RAG & Agents