dead letter queue
A separate queue where messages that keep failing are moved, so they stop blocking others and can be inspected and replayed later.
In more detail
After a set number of failed attempts, the message is parked instead of retried forever. Someone must monitor it; a dead letter queue nobody reads just hides data loss.
Examples
In AWS: A place where events that failed every attempt are stored, so they can be inspected and replayed instead of being lost.Kafka: dlq.send('orders-dlq', key=msg.key, value=msg.value, headers=[('error', str(e).encode())])dlq.send('orders-dlq', key=msg.key, value=msg.value, headers=[('error', str(e).encode())])Related terms
- poison message — A message that fails every time it is processed, for example because it is malformed, and would block a queue if retried forever.
- message queue — A store that holds messages from producers until consumers are ready to process them, decoupling the rate of arrival from the rate of work.
- retry — Sending a request again after it failed or timed out, in the hope that it succeeds the second time.
- replay — Delivering or returning something again: a stored result for a repeated request, or past events for a client that missed them.
- AWS Lambda — The AWS service that runs a piece of code, called a function, each time an event arrives, with no servers for you to manage.
- event — A record of something that has already happened, such as task completed, with an identifier, a time, a type and its data.
- retry topic — A topic that holds records whose processing failed so they can be tried again later without blocking the main topic.
- record header — A key and bytes-value pair attached to a record outside its value, used for metadata such as content type or a trace ID.
- domain event — A record that something meaningful happened in the business, such as an order being created, sent so other services can react.
- assertNever — A small helper function taking a parameter of type never, called in the default branch of a switch to prove every case was handled.
- RabbitMQ — A widely used open-source message broker that routes messages into queues for consumers.
Taught in
- Webhooks Versus Polling — Modern API Architecture: REST, GraphQL & gRPC
- Event-Driven Architecture And Queues — Modern API Architecture: REST, GraphQL & gRPC
- EventBridge Rules: The Rule That Matched Nothing — AWS Certified DevOps Engineer – Professional (DOP-C02) Exam Prep
- Decoupling: SQS, SNS, EventBridge And Step Functions — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Full Mock And Your Review Plan — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Serverless Compute (Lambda) — Cloud Infrastructure & Hyperscaler Architecture
- Blank Names After Every Deploy — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Two Hours Of Scans At Once — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Stuck On One Record — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- A Sink That Died On Record Nine — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Seventy One Percent — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Night Before The Administrator Exam — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Poison Messages And Dead Letter Topics — Apache Kafka & Event Streaming
- Retries With Backoff Topics — Apache Kafka & Event Streaming
- Queues, Events And Asynchronous Work — Scalable System Design & Distributed Architecture
- The Difference Between A Queue And A Log — Scalable System Design & Distributed Architecture
- Exhaustive Narrowing With never In Production Handlers — Production TypeScript
References
- Amazon SQS: Dead-letter queues
- Capturing records of Lambda asynchronous invocations
- Enterprise Integration Patterns: dead letter channel
- RabbitMQ documentation: Dead letter exchanges