OOM killer
The part of the Linux kernel that kills a process when memory runs out.
It kills processes without warning, and without limits it can kill the wrong service.
In more detail
OOM stands for out of memory. When a control group reaches its memory limit, or the whole host runs out, the kernel picks a process and kills it with SIGKILL so the rest can survive. In Docker this shows up as a container that exited with code 137 and OOMKilled set to true in docker inspect. On a host with no limits, the victim may be a different service from the one that leaked.
Examples
docker inspect -f "{{.State.OOMKilled}}" mycontainerkubectl get pod web -o jsonpath=.status.containerStatuses # reason OOMKilled, exit code 137A Node.js process can also die from V8's own "JavaScript heap out of memory" error when the heap reaches --max-old-space-size, before the kernel is involved.Related terms
- SIGKILL — The signal that ends a process immediately; it cannot be caught or ignored.
- memory limit — The most memory a container may use before the kernel kills a process inside it.
- exit code — A number a process returns when it finishes: 0 means success, anything else means something went wrong.
- resource limit — A cap on how much memory, processor time or number of processes a container may use.
Taught in
- Usage, Requests And The Pod That Will Not Fit — Certified Kubernetes Administrator (CKA) Exam Prep
- Requests, Limits And Quotas — Certified Kubernetes Application Developer (CKAD) Exam Prep
- Namespaces And Cgroups: Isolation And Limits — Docker Architecture & Production Containers
- Environment, Secret Files And Runtime Restrictions — Docker Architecture & Production Containers
- Diagnose Crashes, Resource Limits And Unhealthy Containers — Docker Architecture & Production Containers
- Harden And Deliver The Complete Service Stack — Docker Architecture & Production Containers
- Pending All Saturday: How The Scheduler Places Pods — Kubernetes and Cloud Native Associate (KCNA) Exam Prep
- The Status Tells You Where To Look: Troubleshooting Pods — Kubernetes and Cloud Native Associate (KCNA) Exam Prep
- Requests, Limits, Quotas And Horizontal Autoscaling — Kubernetes: Production-Grade Container Orchestration
- Troubleshooting: Pending Pod Triage — Kubernetes: Production-Grade Container Orchestration
- Troubleshooting: OOMKilled, Throttling And Evictions — Kubernetes: Production-Grade Container Orchestration
- Node.js Process Architecture: Memory & Heap Spaces — Node.js Internals & Backend Services
- Runtime Configuration: CLI Flags & Native --env-file — Node.js Internals & Backend Services
- Native Compression: node:zlib & Streaming Codecs — Node.js Internals & Backend Services
- Backpressure Engineering: Handling Mismatched I/O — Node.js Internals & Backend Services
- Production Hardening: Graceful Shutdown & Docker — Node.js Internals & Backend Services
- You Are In Emergency Mode: Boot Failures From GRUB To fstab, On Both Families — CompTIA Linux+ (XK0-006) Exam Prep
- It Is Slow, Says Everyone: CPU, Memory, Disk And Network Bottlenecks — CompTIA Linux+ (XK0-006) Exam Prep
- Ninety Minutes, Five Domains: The Full Mock And The Week Before The Exam — CompTIA Linux+ (XK0-006) Exam Prep
References
- Understanding the Linux Virtual Memory Manager: Out Of Memory Management (kernel.org)
- Resource constraints: memory