thread pool
A fixed or bounded set of reusable worker threads that take tasks from a queue, instead of creating a new thread for each task.
In more detail
Reusing threads avoids the cost of creating them and caps how much work runs at once, protecting memory and the processor. Web servers often serve requests from a pool, so a pool thread blocked on slow I/O is wasted capacity. Size a pool from the expected load and the kind of work it runs.
Examples
ASP.NET Core serves every request from the .NET thread pool.Executors.newFixedThreadPool(2)libuv keeps four threads by default for file system calls, DNS lookups and crypto such as pbkdf2; Piscina provides worker thread pools for CPU-heavy JavaScript.Related terms
- thread — A path of execution inside a process that the operating system schedules on a processor; one process can have many.
- async and await — Language keywords for writing code that waits for slow work, such as a network call, without blocking a thread.
- thread pool starvation — A state where every pool thread is blocked waiting, so new work queues up and the server slows to a crawl though the processor is idle.
- ExecutorService — The standard-library service that owns a pool of worker threads and runs tasks you submit to it, returning a Future for each.
- bounded queue — A waiting line for tasks with a maximum size, so that when it is full new work is rejected instead of piling up in memory.
- common pool — The JVM-wide shared thread pool that parallel streams and CompletableFuture use by default when you do not supply an executor.
- libuv — The C library inside Node.js that runs the event loop, performs asynchronous I/O and manages the thread pool.
- UV_THREADPOOL_SIZE — An environment variable that sets how many threads libuv's thread pool has; the default is 4 and the maximum is 1024.
- worker threads — The Node.js module for running JavaScript in parallel threads inside one process, each with its own V8 isolate and event loop.
- Piscina — A popular npm library that manages a pool of Node.js worker threads, queues tasks for them and returns each result as a promise.
- task queue — A list of jobs waiting to be run, taken off in order by whatever runs them, such as an event loop phase or a pool of worker threads.
- connection pool — A set of already open database connections that a program reuses instead of opening a new one for each request.
- Little's law — A rule from queueing theory: average lead time equals average work in progress divided by average throughput.
- saturation — How full a system’s most constrained resource is, such as its CPU, memory, connections or queue.
- worker — A background process that takes jobs or messages from a queue and does the work, separately from the process that answered the request.
Taught in
- Async and Await: Managing Concurrency — Modern .NET Core, C# & Enterprise Microservices
- Querying, Change Tracking, and Saving Data — Modern .NET Core, C# & Enterprise Microservices
- The ExecutorService — Modern Java: Virtual Threads & High-Throughput Services
- Async With CompletableFuture — Modern Java: Virtual Threads & High-Throughput Services
- Promotion, Wrappers And The Math API — Oracle Certified Professional: Java SE 21 Developer (1Z0-830) Exam Prep
- Strings, StringBuilder And Text Blocks — Oracle Certified Professional: Java SE 21 Developer (1Z0-830) Exam Prep
- Platform And Virtual Threads, Runnable And Callable — Oracle Certified Professional: Java SE 21 Developer (1Z0-830) Exam Prep
- Thread Safety: Locks And Atomics — Oracle Certified Professional: Java SE 21 Developer (1Z0-830) Exam Prep
- Concurrent Collections And Parallel Streams — Oracle Certified Professional: Java SE 21 Developer (1Z0-830) Exam Prep
- libuv Architecture: Event Demultiplexer & Thread Pool — Node.js Internals & Backend Services
- Asynchronous File I/O: node:fs/promises & Handles — Node.js Internals & Backend Services
- Directory Traversal: Recursive Readdir & fs.watch — Node.js Internals & Backend Services
- Worker Threads: worker_threads & SharedArrayBuffer — Node.js Internals & Backend Services
- Thread Pooling: Piscina & CPU-Bound Workload Pools — Node.js Internals & Backend Services
- Observability As A Design Input — Scalable System Design & Distributed Architecture
- Capacity Planning As A Design Input — Scalable System Design & Distributed Architecture
References
- The managed thread pool
- Java Tutorials: Thread Pools
- Java SE 21 API: ThreadPoolExecutor
- libuv: Thread pool work scheduling
- Python documentation: threading module