service time
How long a backend spends actually processing one request, excluding waiting in line.
In more detail
When service times vary, balancing by count leaves some backends with much longer queues than others. Response time is service time plus time spent waiting in a queue.
Related terms
- round robin — A way of spreading work by taking turns in a fixed order, giving the next item to the next recipient in the list.
- queueing — Waiting in line for a busy resource, such as a thread, connection or disk.
Taught in
- Load Balancing And The Failure Modes It Introduces — Scalable System Design & Distributed Architecture