record and replay

Testing code that calls a model by recording real replies once and serving them from a file afterwards, matched by request.

In more detail

A replay makes tests fast, free and deterministic, and a strict replay fails when your code sends a request that was never recorded, printing the difference. Re-record deliberately when the prompt or model changes, and keep live evals for real behaviour.

Related terms

Taught in

References

Certification courses