Least Connections

The previous article ended on a specific gap: round robin distributes requests by count, not by cost, so a backend running unusually slow still receives its full share of new traffic on schedule. Least connections is the direct fix — instead of rotating blindly, it looks at what each backend is doing right now and routes accordingly.

Picking the backend with room to spare

Every time a new request or connection needs a backend, least connections picks whichever one currently has the fewest connections still open, rather than whichever one is simply next in a rotation. NGINX's own documentation for the least_conn directive states the rule plainly: "a request is passed to the server with the least number of active connections."

Backend A: 12 active connections
Backend B: 4 active connections
Backend C: 9 active connections

Next request -> B

That single number — active connections right now — is a genuine, live signal about backend load in a way round robin's fixed rotation never was. A backend that's slow to finish its current requests naturally accumulates more open connections while it works through them, and least connections responds to that accumulation directly by routing less new traffic there, without needing to know why that backend is slow.

Revisiting round robin's blind spot

Go back to the three-backend scenario from the previous article: one backend answering requests at roughly twenty times the latency of its two siblings, with plain round robin still sending it a flat one-third of all traffic regardless. Least connections changes that outcome structurally rather than by luck. A slow backend holds each connection open longer simply because it takes longer to finish responding — which means its active-connection count climbs relative to the two faster backends, and every subsequent routing decision favors the machines finishing requests quickly over the one still working through its backlog.

Round robin asks "whose turn is it"; least connections asks "who's actually free right now." That's the entire difference between the two algorithms, and it's also exactly why least connections costs more to run: the load balancer has to track a live connection count per backend and consult it on every single routing decision, rather than just advancing a pointer through a fixed list.

Where "least connections" stops meaning "least loaded"

The connection count least connections tracks is a proxy for load, not a direct measurement of it, and the two can diverge in a specific, predictable way: a backend serving many cheap, short-lived connections and a backend serving few expensive, long-lived ones can show radically different actual resource usage while looking similar — or even the reverse of what's actually happening — in raw connection count.

A WebSocket-heavy service is the clearest case. WebSocket connections stay open for the entire session, often for hours, while carrying relatively little continuous traffic in between messages. A backend holding a thousand mostly idle WebSocket connections shows a connection count of a thousand; a backend holding ten connections each streaming video at full bandwidth shows a connection count of ten. Plain least connections reads the first backend as vastly more loaded than the second, and routes new traffic away from it accordingly — the exact opposite of what the actual resource usage on each machine calls for.

Practical scenario: least connections making the wrong call under a slow dependency

A checkout service runs behind an NGINX load balancer using least_conn, split across two backends. One backend's outbound connection to the payment provider's API starts responding slowly during a provider-side incident, and instead of that backend receiving less traffic, monitoring shows it receiving more.

upstream checkout_pool {
    least_conn;
    server 10.0.2.10:8080;
    server 10.0.2.11:8080;
}

Checking each backend's live connection count directly, rather than assuming the algorithm is misbehaving, is the right first step:

ss -tn state established '( dport = :8080 )' | wc -l

47

Run against both backends, this shows 10.0.2.10 holding far more established connections than 10.0.2.11 — which, on the surface, looks like exactly what least connections should be avoiding, not causing. The mechanism becomes clear once the payment provider's slowness enters the picture: every checkout request that reaches 10.0.2.10 now takes far longer to complete, because it's blocked waiting on a slow upstream API call, and a request that takes longer to finish holds its connection to the load balancer open longer. That should mean 10.0.2.10's connection count climbs — and it does — but least connections was already sending it a comparable share of new traffic just before the slowdown began, and the newly slow requests are still counted as "active" rather than "finished," compounding rather than draining. The backend isn't overloaded in the CPU sense at all; it's just accumulating in-flight requests stuck waiting on something entirely outside its own control.

This is the least-time gap from the section above showing up concretely: connection count measures "how many requests are currently open," not "how much actual work is happening," and a slow external dependency inflates the first without changing the second. The fix here isn't a load-balancing algorithm change — it's giving the checkout service's calls to the payment provider their own timeout, so a slow provider fails fast and frees the connection instead of holding it (and inflating this backend's apparent load) indefinitely.

Practice exercises

1. Using the definition of least_conn's tie-breaking rule above, explain what happens when two backends both currently show the same lowest connection count.
2. A service handles two very different request types on the same backend pool: a fast JSON API endpoint and a long-lived WebSocket connection used for live notifications. Explain, using the section on WebSocket connections above, why least connections alone might make a poor routing decision here, and name one alternative approach that would do better.
3. In the practical scenario above, would switching from least_conn back to plain round robin have prevented the uneven load? Justify the answer using what each algorithm actually measures.

Least connections and round robin both assume every backend in the pool is roughly equally capable, differing only in how busy each one happens to be at a given moment. That assumption breaks down the moment the pool itself is uneven — a newly added server with twice the CPU and memory of its older neighbors deserves more than an equal share of traffic, and neither algorithm covered so far has any concept of "capacity" built in at all. That's the next article's subject.

Sources

• NGINX, Module ngx_http_upstream_module
• HAProxy Technologies, Configuration Manual