ingestion
Accepting incoming data and getting it safely into storage.
In more detail
A metrics ingest path separates admission, buffering and storage, with a capacity and refusal path at each layer so overload does not become memory exhaustion. Ingestion must absorb bursts without making the sender wait.
Related terms
- admission control — Deciding at the entrance whether a request may start, based on how much work the system can still take.
- bounded queue — A waiting line for tasks with a maximum size, so that when it is full new work is rejected instead of piling up in memory.
- ingest lag — The delay between a data point being produced and it becoming queryable.
Taught in
- Worked Design Three: A Metrics Ingestion Pipeline — Scalable System Design & Distributed Architecture
- Worked Design Three: Scaling The Metrics Pipeline — Scalable System Design & Distributed Architecture