eval gate

An eval suite run in CI that fails the build when scores fall below thresholds, so a regression cannot ship unnoticed.

In more detail

A gate is only useful if a red build means a real regression. Keep pull request builds deterministic with recorded replies, re-record deliberately when the prompt or model changes, and run live evals on a schedule.

Related terms

Taught in

References

Certification courses