trials

Running each eval case several times to measure how much scores vary between runs of the same system.

In more detail

Model output varies, so one run cannot tell a regression from noise. Measure the current release over several trials, set each threshold below the baseline by the noise you saw, and require critical cases to pass every trial.

Related terms

Taught in

References

Certification courses