Multiple servers and resource pooling

Two servers sharing one waiting line can use idle capacity more effectively than two randomly assigned lines. Pooling reduces occasions when one server is idle while another has a queue. It does not turn two 10 ms servers into one 5 ms server.

State-dependent capacity in M/M/c

Assume Poisson arrivals, identical servers with independent exponential service at rate $\mu$, one FCFS queue, no abandonment and unlimited waiting space. With $n$ jobs inside, the total completion rate is $\min (n,c)\mu$.

Define offered load $a=\lambda /\mu$ and per-server utilization $\rho =a/c$. Stationarity requires $\rho <1$. Adjacent-state balance yields:

$${\pi }_0={\left[\sum _{n=0}^{c-1}\frac{a^n}{n!}+\frac{a^c}{c!(1-\rho )}\right]}^{-1}.$$

An arrival waits exactly when it sees at least $c$ jobs. This probability is Erlang C:

$$C(c,a)=\frac{a^c}{c!(1-\rho )}{\pi }_0,E[W_q]=\frac{C(c,a)}{c\mu -\lambda },E[T]=E[W_q]+\frac{1}{\mu }.$$

The waiting tail is $P(W_q>t)=C(c,a)e^{-(c\mu -\lambda )t}$. Total time also includes the arriving job's own service, so it is generally wrong to assign total time an exponential rate $c\mu -\lambda$.

Pooling versus splitting with equal resources

For two servers at $\mu =100$/s and total $\lambda =160$/s, $a=1.6$, $\rho =0.8$, ${\pi }_0=1/9$ and $C=32/45\approx 0.7111$. Pooled mean waiting is 17.78 ms and mean system time 27.78 ms.

If each arrival independently selects a server with equal probability, Poisson splitting produces two M/M/1 queues at 80/s each. Each has mean wait 40 ms and mean system time 50 ms. Hardware is unchanged; coordinating idle capacity changes waiting.

Start with two servers and vary $\rho$, then vary server count. Changing $c$ also changes offered traffic through $\lambda =c\mu \rho$. This studies scale at fixed per-server utilization, not scaling under fixed external traffic. At $c=1$, the curves coincide, providing a boundary check.

Pooling has implementation conditions

The shared queue must dispatch promptly, and servers must be interchangeable for the jobs. Locality, accelerator type or access restrictions can prevent arbitrary assignment. Shared-queue locks, communication and cache effects are absent from M/M/c. Random splitting is only a baseline: shortest-queue routing, least connections and work stealing differ from it.

Erlang B describes loss systems without waiting space; Erlang C holds arrivals until service becomes available. Abandonment motivates Erlang A. General service times require M/G/c approximations or simulation. These names do not label interchangeable formulas.

Check your understanding

1. Why not replace two 100/s servers with a single M/M/1 server at 200/s?

2. Eight workers share a database allowing only two concurrent operations. Is M/M/8 appropriate?

Further reading

NPTEL's multiple-server lecture derives Erlang C from state-dependent completion rates. Its offered-load notation differs: this series consistently uses $a=\lambda /\mu$ and $\rho =a/c$.