test suite
A collection of tests run together to check a piece of software.
In more detail
A good suite is not the one with the most assertions; it is the one that fails for a fault a user would notice and points to a useful repair. Suites are usually run automatically on every change.
Examples
python3 contract_test.py -k NotFoundThe capstone test harness wires routing, sessions, the database, outboxes and tracing into one suite.In this course: A suite is only useful if people run it, so its speed matters: a suite that takes twenty minutes gets run once a day instead of on every change. Parts of a suite can be selected by file, name, keyword or marker so that a fast subset runs constantly and slow tests run less often.Related terms
- contract testing — Testing that an API’s real behaviour matches its written contract, and that its callers only rely on what the contract promises.
- unit test — An automated check of one small piece of code, such as a single function, that runs quickly and on its own.
- integration test — A test that checks several real parts working together, such as the service talking to a real database.
- end-to-end test — A test that drives the whole system from the outside, often through a browser or public API, with every part running.
- test pyramid — A guide for balancing a test suite: many fast unit tests, fewer integration tests and a small number of end-to-end tests.
- describe() and it() — Test functions that group tests into suites (describe) and define individual test cases (it).
- automated test — A small program that runs part of your code and checks that it did what you claimed it would.
- test runner — A tool that finds test files, runs the tests in them and reports which passed and which failed.
- marker — A label attached to a test with @pytest.mark.name so you can select, skip or configure tests by that label.
- pytest -k — The pytest option that runs only tests whose names match a substring or a small expression using and, or and not.
Taught in
- HTTP Methods And Status Codes — Modern API Architecture: REST, GraphQL & gRPC
- Problem Details: RFC 9457 — Modern API Architecture: REST, GraphQL & gRPC
- Unit And Integration Testing — Modern API Architecture: REST, GraphQL & gRPC
- Contract Testing — Modern API Architecture: REST, GraphQL & gRPC
- Trunk-Based Development Versus Git Flow — DevOps & Site Reliability Engineering
- Testing Strategies And The Test Pyramid — DevOps & Site Reliability Engineering
- End-to-End Testing with Playwright: Multi-Tier Suites — Production Full-Stack Engineering
- Full-Stack System Verification: Capstone Integration — Production Full-Stack Engineering
- Release Trains And Long Term Support — Modern Java: Virtual Threads & High-Throughput Services
- Testing Basics With JUnit 5 — Modern Java: Virtual Threads & High-Throughput Services
- Database Integration With Testcontainers — Modern Java: Virtual Threads & High-Throughput Services
- Native Test Runner: node:test & node:assert/strict — Node.js Internals & Backend Services
- The Cost And Value Of A Test — Automated Testing, TDD & Quality Engineering
- The Test Pyramid And Where It Is Wrong — Automated Testing, TDD & Quality Engineering
- Introducing pytest — Automated Testing, TDD & Quality Engineering