service time

How long a backend spends actually processing one request, excluding waiting in line.

In more detail

When service times vary, balancing by count leaves some backends with much longer queues than others. Response time is service time plus time spent waiting in a queue.

Related terms

Taught in

References