transactional outbox
Saving an event into an outbox table in the same database transaction as the business change, then having a separate worker publish it.
In more detail
Because the order row and its outbox row commit or roll back together, an event exists if and only if the change happened. A relay worker polls the outbox and publishes each event to the broker, retrying until it succeeds, which gives at-least-once delivery.
Related terms
- dual write — Writing the same change to two separate systems, such as a database and a message broker, from application code with no shared transaction.
- at-least-once delivery — A delivery guarantee under which every message arrives, but some may arrive more than once.
- idempotency — The property of an operation that gives the same result whether you run it once or many times, because a repeat run finds nothing left to do.
- background worker — A process that runs outside the request path and handles work such as publishing events or sending emails.
- outbox relay — The process that reads unpublished rows from an outbox table, publishes them to a message broker such as Kafka and then marks them as published.
- transaction — A group of database changes that succeed or fail together, so the data is never left half-updated.
- change data capture — Reading every insert, update and delete from a database's own change log and turning them into a stream of events.
- deduplication — Detecting and discarding repeated copies of the same message or request, usually by remembering identifiers already processed.
Taught in
- Distributed Cache Invalidation: Redis & Tagged Purging — Production Full-Stack Engineering
- Asynchronous Event Processing: Outbox Pattern & Queues — Production Full-Stack Engineering
- Full-Stack System Verification: Capstone Integration — Production Full-Stack Engineering
- Dual Writes And Why They Lose Data — Apache Kafka & Event Streaming
- The Transactional Outbox — Apache Kafka & Event Streaming
- Idempotency, And Exactly Once As A Fiction — Scalable System Design & Distributed Architecture