Little’s law and measurement boundaries

Little's law, $L=\lambda E[T]$, connects average concurrency, flow rate and mean residence time. Its strength comes from consistent accounting boundaries. It requires neither Poisson arrivals nor an exponential service distribution, and its mean cannot be replaced by p99.

Why area equals accumulated residence time

Within $[0,H]$, job $i$ contributes the indicator $1\{a_i\le t<d_i\}$ to occupancy. Sum over jobs and integrate:

$${\int }_0^{H}N(t)dt=\sum _i\left|[a_i,d_i)\cap [0,H]\right|.$$

This is a sample-path identity without probability assumptions. Both sides have units job-seconds. Intersections include the part of any job that arrived before the window or finishes after it.

If the system is empty at both endpoints and all $m$ arrivals depart within the window:

$${\bar{N}}_H=\frac{1}{H}\sum _{i=1}^m{T}_i=\frac{m}{H}\cdot \frac{1}{m}\sum _{i=1}^m{T}_i.$$

For a stable system with well-defined long-run rates and mean residence times, and suitable conditions making residual boundary contributions divided by $H$ vanish, this gives $L=\lambda E[T]$. Stationarity is a common analytical setting, not a prerequisite for the finite-window area identity. A short interval that looks calm does not establish the required limits.

What a cutoff leaves out

The experiment uses arrivals $[0,1,2,6]$ s and services $[3,1,2,1]$ s. At $H=5$ s, two jobs have finished. Occupancy area is $3+3+3=9$ job-seconds, so average concurrency is 1.8. Completion rate is $2/5$ and completed-job mean residence is 3 s; their product is only 1.2. The missing 3 job-seconds belong to the third, unfinished request.

Move the endpoint to 7 s or later. All jobs have finished and the boundary contribution becomes zero. Extending the window further adds idle time, reducing both estimates together. Observation length is part of the measurement definition.

Match boundary, population and time range

A stable pool accepting 200 requests/s with mean connection holding time 40 ms has mean occupied connections $200\times 0.040=8$. This does not say that configuring eight connections meets a waiting target: average occupancy leaves no explicit allowance for fluctuations.

For the waiting area, use $L_q=\lambda E[W_q]$; for the entire system, use $L=\lambda E[T]$. If only 800/s of 1000/s offered requests enter, use 800/s with internal occupancy. When exits include failures or cancellations, residence statistics must include the same exit population. Successful throughput cannot be paired directly with occupancy of all attempts.

Retries create multiple attempts per business request. A business boundary can span first submission to final outcome; an attempt boundary tracks each entry and exit separately. Either is useful, but their counts and durations cannot be mixed.

Check your understanding

1. Average concurrency is 50 and throughput 500/s. Is p99 therefore 100 ms?

2. During an incident, occupancy rises but completed-job mean latency stays low. Has Little's law failed?

Further reading

The MIT 2026 queueing lecture discusses Little's law and subsystem boundaries. The cutoff example above follows directly from its independently constructed timestamps.