---
title: 'M/M/1: utilization and tail latency'
url: https://doc.liz6.com/en/theory/03-queueing-theory/04-mm1-and-tail-latency
locale: en
area: theory
tags:
- Queueing theory
- Theory
date: 2026-09-10
modified: 2026-09-10
description: M/M/1 is a fully solvable model of random arrivals and service. It explains nonlinear waiting near full utilization and shows why a mean alone cannot represent every user's experience.
---
# M/M/1: utilization and tail latency
M/M/1 is a fully solvable model of random arrivals and service. It explains nonlinear waiting near full utilization and shows why a mean alone cannot represent every user's experience.
## From state balance to means
Assume exogenous Poisson arrivals at $\lambda$, IID exponential service at $\mu$, mutual independence, one FCFS server, unlimited waiting space, and no abandonment or retries. State $n$ counts all jobs inside. Arrivals increase it at rate $\lambda$; when nonempty, completions decrease it at rate $\mu$.
Stationary adjacent-state balance is $\pi_n\lambda=\pi_{n+1}\mu$. With $\rho=\lambda/\mu$, normalizing the geometric series yields:
$$\pi_n=(1-\rho)\rho^n,\quad\rho<1;\qquad L=\sum_{n\ge0}n\pi_n=\frac{\rho}{1-\rho}.$$
Consequently:
$$E[T]=\frac1{\mu-\lambda},\quad E[W_q]=\frac{\rho}{\mu-\lambda},\quad L_q=\frac{\rho^2}{1-\rho}.$$
At $\rho\ge1$ the infinite-state chain has no normalizable stationary distribution. Continuing the formula to obtain a negative wait is invalid. Equality is not a safe equilibrium for this stochastic queue.
## An exactly computable tail
An arrival sees $n$ other jobs with probability $\pi_n$. Memoryless service makes its system time the sum of $n+1$ independent exponential stages. Mixing these sums with geometric weights gives:
$$P(T>t)=e^{-(\mu-\lambda)t},\qquad t\ge0.$$
Waiting has a different distribution: probability $1-\rho$ corresponds to immediate service.
$$P(W_q>t)=\rho e^{-(\mu-\lambda)t},\qquad P(W_q=0)=1-\rho.$$
The system-time $p$ quantile is $t_p=-\ln(1-p)/(\mu-\lambda)$. Thus p99 is approximately 4.605 times the mean in this model. That ratio is not a universal production rule.
**M/M/1: utilization and tail latency · Experiment**
Stationary FCFS M/M/1, unlimited waiting space, μ=100/s. The Wq tail equals ρ at t=0 because other arrivals wait zero time. The lower marker is current utilization. All values are exact within this model.
At $\mu=100$/s and $\rho=0.8$, mean system time is 50 ms, mean wait 40 ms and system-time p99 about 230 ms. At $\rho=0.9$, they become 100, 90 and 461 ms. Mean service remains 10 ms throughout.
## Derive a conditional capacity boundary
Requiring system-time p99 at most 200 ms in this model gives:
$$\mu-\lambda\ge\frac{\ln100}{0.2}\approx23.026\ \mathrm{s}^{-1}.$$
With $\mu=100$/s, this permits $\lambda\le76.974$/s. The result is conditional on the model. Long services, batching, several stages or a slowing dependency require another model and validation with load tests.
A utilization curve also does not capture every scaling effect. More workers may leave the bottleneck's $\mu$ unchanged; contention may even reduce it. Identify service occupancy and load-dependent behavior before treating service capacity as constant.
## Check your understanding
1. At 50% utilization, do half of requests have zero total latency?
Reasoning
Half can start without waiting, but they still need service. $W_q$ has a probability mass at zero; $T$ does not. Waiting probability is not the total-response distribution.
Reasoning
Only if the stationary FCFS M/M/1 assumptions apply. Real response times may be mixtures, multistage sums or dependent processes. Record the distribution and check model error instead of assuming this ratio.