concurrency
Having several pieces of work in progress during overlapping periods of time, such as requests being handled at once.
In more detail
Concurrent tasks need not run at the same instant: a single thread can switch between tasks whenever one is waiting for input or output, while truly simultaneous execution on several processor cores is called parallelism. Tasks that share data can interleave in unpredictable ways, so shared state needs protection such as locks, transactions or atomic operations. Bounding concurrency, for example with a fixed worker pool, stops a busy service from exhausting a dependency.
Examples
Go: one goroutine per health-check targetJava: threads, ExecutorService and CompletableFuture are the main tools.Node.js achieves concurrency on its one main thread through the event loop.A database lets many readers and writers work at once without corrupting data, which a shared file cannot.Related terms
- load testing — Measuring how a system behaves while a chosen amount of traffic is sent to it, to find its latency, throughput and limits.
- saturation — How full a system’s most constrained resource is, such as its CPU, memory, connections or queue.
- worker — A background process that takes jobs or messages from a queue and does the work, separately from the process that answered the request.
- Little's law — A rule from queueing theory: average lead time equals average work in progress divided by average throughput.
- bounded concurrency — Capping how many requests a caller will have in progress to a dependency at once.
- race condition — A bug whose outcome depends on the timing of two things happening at once.
- process formation — The set of process types an app runs and how many of each are running.
- process type — A named kind of work an app does, such as web for requests, worker for queued jobs or clock for scheduled tasks.
- thread — A path of execution inside a process that the operating system schedules on a processor; one process can have many.
- horizontal scaling — Handling more load by running more copies of a process, often on more machines, rather than making one machine bigger.
- goroutine — A function running concurrently with the rest of a Go program, started by putting the go keyword before a call.
- channel — A typed pipe in Go that one goroutine sends values into and another receives them from, safely handing data between them.
- data race — A bug where two goroutines access the same memory at the same time and at least one of them writes, without any synchronization.
- mutex — A lock that lets only one piece of code at a time run a critical section or do a particular piece of work.
- sync.WaitGroup — A counter from Go's sync package that lets one goroutine wait until a group of other goroutines have all finished.
- ExecutorService — The standard-library service that owns a pool of worker threads and runs tasks you submit to it, returning a Future for each.
- CompletableFuture — A Future that can be completed later and chained, letting you describe asynchronous steps such as transform, combine and recover without nesting callbacks.
- parallel stream — A stream that splits its work across several threads from the common pool, which can speed up large computations and slow down small ones.
- parallelism — Doing several pieces of work at the same time instead of one after another.
- event loop — The central loop of an asynchronous program that waits for events, such as timers firing or data arriving, and runs the matching callbacks.
- asynchronous I/O — Starting an input or output operation and carrying on with other work, being told later, usually through a callback or promise, when it has finished.
- transaction — A group of database changes that succeed or fail together, so the data is never left half-updated.
- isolation level — A setting that controls how much concurrent transactions can see of each other's changes and how they may interleave.
- lost update — A concurrency bug where two sessions read the same value, both write a result based on it, and the later write silently erases the earlier one.
Taught in
- Load Testing — Modern API Architecture: REST, GraphQL & gRPC
- Performance Optimization — Modern API Architecture: REST, GraphQL & gRPC
- Observability: Logs, Metrics, Traces — Modern API Architecture: REST, GraphQL & gRPC
- Serverless And Container Patterns: Where The Limits Bite — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Load Balancers And Auto Scaling: Capacity That Arrives On Time — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Compute Performance: Families, Placement And Lambda Memory — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Database Performance: Caches, Proxies And Capacity Units — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- The Full Mock And Your Review Plan — AWS Certified Solutions Architect – Associate (SAA-C03) Exam Prep
- Config And Backing Services — DevOps & Site Reliability Engineering
- Port Binding And Concurrency — DevOps & Site Reliability Engineering
- Disposability, Dev Prod Parity And Logs — DevOps & Site Reliability Engineering
- What Go Is and Why It Matters — Concurrent Systems Programming with Go
- Goroutines and Concurrent Execution — Concurrent Systems Programming with Go
- The select Statement — Concurrent Systems Programming with Go
- Context Cancellation and Timeouts — Concurrent Systems Programming with Go
- Project Setup and Configuration — Concurrent Systems Programming with Go
- Executing Concurrent Checks — Concurrent Systems Programming with Go
- Containerizing and Testing the Final Build — Concurrent Systems Programming with Go
- Concurrency Basics: Threads And Runnable — Modern Java: Virtual Threads & High-Throughput Services
- The ExecutorService — Modern Java: Virtual Threads & High-Throughput Services
- Async With CompletableFuture — Modern Java: Virtual Threads & High-Throughput Services
- libuv Architecture: Event Demultiplexer & Thread Pool — Node.js Internals & Backend Services
- Worker Threads: worker_threads & SharedArrayBuffer — Node.js Internals & Backend Services
- Thread Pooling: Piscina & CPU-Bound Workload Pools — Node.js Internals & Backend Services
- Why Tables And Relations Beat Files — PostgreSQL, Indexing & Relational Data Modeling
- Isolation Levels And The Lost Update — PostgreSQL, Indexing & Relational Data Modeling
- Where PostgreSQL Differs From SQLite — PostgreSQL, Indexing & Relational Data Modeling
- Cache Invalidation And The Three Hard Cases — Scalable System Design & Distributed Architecture
- Idempotency, And Exactly Once As A Fiction — Scalable System Design & Distributed Architecture
- Load Balancing And The Failure Modes It Introduces — Scalable System Design & Distributed Architecture
- Timeouts, And Avoiding Cascaded Failure — Scalable System Design & Distributed Architecture
- Capacity Planning As A Design Input — Scalable System Design & Distributed Architecture
- Worked Design One: Ticket Booking, Correctness First — Scalable System Design & Distributed Architecture
References
- Grafana k6: Load testing
- Google SRE book: Handling overload
- Operating Systems: Three Easy Pieces, Concurrency: An Introduction
- John D. C. Little: Little's Law as Viewed on Its 50th Anniversary (Operations Research, 2011)
- The Twelve-Factor App: VIII. Concurrency
- Effective Go: concurrency
- A Tour of Go: concurrency
- Java Tutorials: Concurrency
- Node.js: Overview of blocking vs non-blocking (Concurrency and throughput)
- PostgreSQL: concurrency control
- SQLite: isolation in SQLite