flaky test

A test that sometimes passes and sometimes fails on the same code, because it depends on something it does not control.

In more detail

Common causes are time, run order, randomness, networks and shared state. Retrying hides the evidence and slows the build; instead reproduce the failure, name the moving input and make it explicit or isolated. Running the test repeatedly, alone and in a shuffled suite, helps find the cause.

Example

Visual tests are flaky when a screenshot catches an animation mid-way, a font still loading or different text rendering; freeze animations, await fonts, fix the viewport and run in one container.

Related terms

Taught in

References

Certification courses