eval-driven development

Deciding what a good output looks like and writing evals for it before changing an AI feature, then running them on every change.

In more detail

It is test-driven development for probabilistic systems: define success criteria first, design outputs so code can grade them, and let the eval results, not impressions, decide whether a change ships. Every production failure becomes a new case before the fix is written.

Related terms

Taught in

References

Certification courses