regression
A bug where something that used to work stops working, or works worse, after a change.
In more detail
Regressions usually come from small, plausible changes to code that other parts of the system rely on. A regression test is any test that would fail if the old, correct behaviour came back broken. Performance can regress too, and averages can hide a latency regression that percentiles reveal. Catching regressions the moment they are introduced is the main thing a test suite buys you.
Example
A visual regression is an unintended change in appearance, such as clipped text after a line-height tweak.Related terms
- visual regression testing — Testing that takes screenshots of components and compares them pixel by pixel with approved baseline images to catch unintended visual changes.
- unit test — An automated check of one small piece of code, such as a single function, that runs quickly and on its own.
- quality gate — A check, or set of checks, that a change must pass before it can move on, such as being merged or promoted to the next pipeline stage.
- regression test — A test run to check that behaviour which used to work still works after a change.
- percentile — The value below which a given percentage of measurements fall; p99 latency is the time 99 in 100 requests beat.
- automated test — A small program that runs part of your code and checks that it did what you claimed it would.
- test suite — A collection of tests run together to check a piece of software.
- continuous integration — Automatically building and testing every change as soon as it is pushed, so problems are caught early.
Taught in
- Structured Output — AI Engineering: LLM Apps, RAG & Agents
- Prompts As Code — AI Engineering: LLM Apps, RAG & Agents
- Memory And Compaction — AI Engineering: LLM Apps, RAG & Agents
- Measuring Retrieval — AI Engineering: LLM Apps, RAG & Agents
- Eval-Driven Development — AI Engineering: LLM Apps, RAG & Agents
- Evaluating RAG And Agents — AI Engineering: LLM Apps, RAG & Agents
- Regression Tests In CI — AI Engineering: LLM Apps, RAG & Agents
- Automated Accessibility Testing: axe-core & Testing Library — Design Systems & Component Engineering
- Storybook & Component Driven Development Workflows — Design Systems & Component Engineering
- Visual Regression Testing: Playwright & Snapshot Differencing — Design Systems & Component Engineering
- Decoupling Release From Deployment: Feature Flags — DevOps & Site Reliability Engineering
- Percentiles Rather Than Averages — DevOps & Site Reliability Engineering
- Service Level Objectives — DevOps & Site Reliability Engineering
- The Cost And Value Of A Test — Automated Testing, TDD & Quality Engineering
References
- Playwright: visual comparisons
- ISTQB glossary: regression testing
- Martin Fowler: self-testing code
- pytest: get started