throughput
How much work a system completes per unit of time, such as requests per second.
In more detail
Throughput and latency describe different pressures: a service can complete many requests per second while each one is slow. As a system saturates, latency usually climbs sharply before throughput stops growing.
Examples
Kafka producers raise throughput with batching, linger.ms and compression.Throughput group: change lead time, deployment frequency, failed deployment recovery time.Related terms
- latency — The time between asking for something and getting the answer back.
- saturation — How full a system’s most constrained resource is, such as its CPU, memory, connections or queue.
- load testing — Measuring how a system behaves while a chosen amount of traffic is sent to it, to find its latency, throughput and limits.
- Amazon EFS — Amazon Elastic File System, a managed shared filesystem that many Linux instances, containers and functions can mount at once.
- EBS volume — One Amazon EBS disk, created in one availability zone and attached to an EC2 instance.
- record batch — A group of records sent to the broker together in one request, instead of one request per record.
- linger.ms — The producer setting for how many milliseconds to wait for more records before sending a batch that is not yet full.
- parallelism — Doing several pieces of work at the same time instead of one after another.
- instability — DORA's name for the group of metrics that measure how well deployments go: change fail rate and deployment rework rate.
- DORA metrics — Five measures of software delivery from the DORA research: change lead time, deployment frequency, failed deployment recovery time, change fail rate and deployment rework rate.
- Little's law — A rule from queueing theory: average lead time equals average work in progress divided by average throughput.
- async and await — Language keywords for writing code that waits for slow work, such as a network call, without blocking a thread.
- thread pool — A fixed or bounded set of reusable worker threads that take tasks from a queue, instead of creating a new thread for each task.
- benchmark — A repeatable measurement of how long code takes, or how much it uses, under a fixed workload, or more broadly a standard point of comparison such as an industry average.
- queries per second — The number of requests a system handles each second, the usual unit for traffic and capacity.
- bandwidth — How much data a network connection can carry per second, and by extension the amount of data a request or response uses up.
Taught in
- Load Testing — Modern API Architecture: REST, GraphQL & gRPC
- Performance Optimization — Modern API Architecture: REST, GraphQL & gRPC
- Ten Findings, Six Pillars — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- One Database Doing Five Jobs — AWS Certified Cloud Practitioner (CLF-C02) Exam Prep
- Decoupling: SQS, SNS, EventBridge And Step Functions — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Load Balancers And Auto Scaling: Capacity That Arrives On Time — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Storage Performance: IOPS, Throughput And The Right Volume — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Compute Performance: Families, Placement And Lambda Memory — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Performance Clinic: Five Slow Systems, Five Fixes — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Database Cost: Serverless, Capacity Modes And Reservations — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Full Mock And Your Review Plan — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Elastic Compute Cloud (EC2) — Cloud Infrastructure & Hyperscaler Architecture
- Object, Block, and File Storage — Cloud Infrastructure & Hyperscaler Architecture
- Elastic File System (EFS) — Cloud Infrastructure & Hyperscaler Architecture
- Where A Record Lands — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Retry That Overtook — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Alert That Came An Hour Late — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- The Setting That Came Back — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- Full Disks On Black Friday — Confluent Apache Kafka Certifications (CCDAK and CCAAK) Exam Prep
- What DevOps Actually Is — DevOps & Site Reliability Engineering
- Throughput: Change Lead Time And Deployment Frequency — DevOps & Site Reliability Engineering
- Restoration: Failed Deployment Recovery Time — DevOps & Site Reliability Engineering
- The Four Keys History And Retiring MTTR — DevOps & Site Reliability Engineering
- SDLC: From Request To Running Software — DevOps & Site Reliability Engineering
- Async and Await: Managing Concurrency — Modern .NET Core, C# & Enterprise Microservices
- Batching, Linger And Throughput — Apache Kafka & Event Streaming
- Bytes On The Wire: JSON, Avro And Protobuf — Apache Kafka & Event Streaming
- libuv Architecture: Event Demultiplexer & Thread Pool — Node.js Internals & Backend Services
- Buffer Fundamentals: Slab Allocator & V8 Memory — Node.js Internals & Backend Services
- Writable Streams: Drain Events & highWaterMark — Node.js Internals & Backend Services
- Back Of The Envelope Estimation: Traffic And Throughput — Scalable System Design & Distributed Architecture
- Back Of The Envelope Estimation: Storage And Bandwidth — Scalable System Design & Distributed Architecture
- Replication, Failover And Leader Election — Scalable System Design & Distributed Architecture
- Queues, Events And Asynchronous Work — Scalable System Design & Distributed Architecture
- Capacity Planning As A Design Input — Scalable System Design & Distributed Architecture
- Worked Design One: Ticket Booking, Correctness First — Scalable System Design & Distributed Architecture
- Worked Design One: Scaling The Ticket Booking System — Scalable System Design & Distributed Architecture
- Worked Design Two: Scaling Chat Fan Out — Scalable System Design & Distributed Architecture
References
- Google SRE book: Monitoring distributed systems
- Grafana k6: Built-in metrics
- Amazon EFS performance specifications
- Apache Kafka design: efficiency
- DORA's software delivery performance metrics
- ASP.NET Core best practices
- Node.js: Overview of blocking vs non-blocking (Concurrency and throughput)
- MDN Web Docs glossary: Bandwidth
- Google SRE workbook: Managing Load