false positive
A check result that claims something is true when it is not, such as a test passing on broken code or a tool flagging correct code.
In more detail
False positives are the price of fast or approximate checks: a filter, scanner or test reports something that is not really there. Too many of them erode trust in the check, although some designs accept a small, known rate in exchange for large savings.
Examples
A Bloom filter says "probably present" for an absent item when all of its bits happen to be set already.False positives waste analyst time and cause alert fatigue.In this course: In testing the dangerous kind is a green test that lets a broken behaviour through, often caused by over-mocking or a missing match argument. In automated review, tools that raise false alarms nobody can fix lose the team's trust in the gate.Related terms
- false negative — A result that wrongly says "no" when the true answer is "yes".
- Bloom filter — A compact, probabilistic set that can say "definitely not present" or "probably present", using a bit array and several hash functions.
- security gate — An automated check in a pipeline that stops a change going further when it fails, usually by exiting with a non-zero code.
- secret scanning — Searching code, commits and files for credentials such as keys and tokens, ideally before they are committed.
- anomaly detection — Learning what normal behaviour looks like for a workload or identity and flagging what is new.
- true positive — An alert that fired on real malicious activity: the rule was right and the activity was an attack.
- benign true positive — An alert where the rule correctly spotted the behaviour it looks for, but the activity turned out to be allowed.
- alert fatigue — The state where people receive so many unhelpful alerts that they start ignoring them, including the real ones.
- tuning — Adjusting a detection rule so it raises fewer false alerts without missing real attacks.
- finding — One problem reported by a scanner or review, with a location, a rule and a reason.
- suppression — Marking one diagnostic or scanner finding as intentionally ignored for some code, usually with an inline comment or attribute and a reason.
- over-mocking — Replacing so many collaborators with mocks that tests only check the script the test writer invented, not real behaviour.
- quality gate — A check, or set of checks, that a change must pass before it can move on, such as being merged or promoted to the next pipeline stage.
- pytest.raises — A pytest helper, used in a with block, that makes a test pass only if the code inside raises the named exception.
Taught in
- Bloom Filters: Membership Without The Data — Algorithms & Data Structures for Working Engineers
- Two Hundred And Fourteen Findings: Vulnerability Management — CompTIA Cloud+ (CV0-004) Exam Prep
- Pre-Commit Secret Scanning with Gitleaks & Entropy — Cloud Security & DevSecOps Engineering
- Static Application Security Testing with Semgrep — Cloud Security & DevSecOps Engineering
- Runtime Anomaly Detection with Falco & eBPF — Cloud Security & DevSecOps Engineering
- Static IaC Scanning: Linting Terraform with Checkov — Cloud Security & DevSecOps Engineering
- Threat Detection with GuardDuty & Anomaly Analytics — Cloud Security & DevSecOps Engineering
- Threat Actors, Motivations & Cyber Kill Chain — Cybersecurity Fundamentals, GRC & Cryptography
- Defense-in-Depth & Security Control Categories — Cybersecurity Fundamentals, GRC & Cryptography
- Regulatory Compliance: SOC 2, HIPAA, GDPR & PCI-DSS — Cybersecurity Fundamentals, GRC & Cryptography
- Data Loss Prevention & Egress Monitoring — Cybersecurity Fundamentals, GRC & Cryptography
- SOC Operating Models, Tiers & Key Performance Metrics — Security Operations & Threat Hunting (SOC)
- Alert Triage, False Positive Reduction & Fatigue Mitigation — Security Operations & Threat Hunting (SOC)
- Event Correlation Rules, Thresholds & Anomaly Pipelines — Security Operations & Threat Hunting (SOC)
- Detection as Code: Authoring & Testing Sigma Rules — Security Operations & Threat Hunting (SOC)
- YARA Rule Authoring & File Signature Matching — Security Operations & Threat Hunting (SOC)
- Static Application Security Testing — Application Security & Threat Modeling for Engineers
- Secret Scanning — Application Security & Threat Modeling for Engineers
- Why Over-Mocking Breeds False Positives — Automated Testing, TDD & Quality Engineering
- What To Automate Away — Automated Testing, TDD & Quality Engineering
References
- NIST Dictionary of Algorithms and Data Structures: Bloom filter
- NIST glossary: false positive
- Martin Fowler: mocks aren't stubs