Skip to content

Round Robin

Three backends behind a load balancer, no configuration beyond pointing it at them, and requests already spreading out evenly across all three — round robin is what almost every load balancer does when nobody has told it to do anything more specific. NGINX documents it as the default upstream method precisely for that reason: it needs no directive to enable, unlike least_conn or hash, which have to be requested explicitly.

The algorithm itself

Round robin cycles through the backend list in a fixed order, sending each new connection to the next entry and wrapping back to the start once it reaches the end — a rotation, not a decision based on anything the backends are actually doing.

Backends: [ A, B, C ]

Request 1 -> A
Request 2 -> B
Request 3 -> C
Request 4 -> A
Request 5 -> B

That's the entire algorithm. It needs no information about how busy a backend currently is, how long its last request took, or how powerful the underlying hardware is — which is exactly why it's cheap to implement and predictable to reason about, and also exactly why it breaks down the moment those things stop being roughly equal across the pool.

Where the rotation stops being fair

Round robin's guarantee is about count, not about load. It promises every backend gets an equal number of requests over time; it says nothing about whether those requests are equally expensive to handle.

Picture a product search endpoint and a health-check endpoint sitting behind the same three-backend pool. A search request might take 400 milliseconds and touch the database; a health check returns in under a millisecond. Round robin sends both kinds of request through the identical rotation, with no awareness that they cost wildly different amounts of backend time:

Request 1 (search, 400ms)   -> A
Request 2 (health, <1ms)    -> B
Request 3 (search, 400ms)   -> C
Request 4 (search, 400ms)   -> A
Request 5 (health, <1ms)    -> B

Over a large enough sample, if search requests happen to arrive more often when it's A's or C's turn than B's, those two backends end up carrying measurably more actual work than B does — despite each having received an equal share of total requests. Round robin already did its job by that measure: three backends, five requests, a perfectly even 1.67 requests apiece if the rotation continued. It just never promised anything about the cost of what it distributed, and a CPU graph showing two backends running hot while a third idles is the visible symptom of that gap, not a bug in the rotation itself.

DNS round robin: the same name, a different mechanism

The A record article already touched on a superficially similar idea — a domain name resolving to multiple A records, with resolvers picking among them — as a contrast to a real load balancer. It's worth being precise about why that isn't the same mechanism as this article's subject, even though both get called "round robin."

A DNS resolver returning multiple addresses has no ongoing connection to either backend and no way to know if the one it just handed out is even still running — it hands out an address and its job ends there. The load balancer's round robin described above sits in the actual traffic path, sees every connection, and works against a pool that's continuously health-checked, the way the load balancer article covered. DNS round robin distributes lookups; a load balancer's round robin distributes live traffic against a pool it's actively watching — the shared name describes the rotation pattern, not a shared architecture.

Practical scenario: three backends, one of them twice as slow

A team runs three backend instances behind an NGINX load balancer configured with plain round robin, and the p99 latency for the whole service periodically spikes even though CPU usage on each backend looks unremarkable in isolation.

upstream backend_pool {
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

Checking each backend's own request duration, rather than only the aggregate figure the monitoring dashboard shows by default, tells a different story than the CPU graphs did:

curl -o /dev/null -s -w '%{time_total}\n' http://10.0.1.10:8080/health
curl -o /dev/null -s -w '%{time_total}\n' http://10.0.1.11:8080/health
curl -o /dev/null -s -w '%{time_total}\n' http://10.0.1.12:8080/health

0.014
0.013
0.312

10.0.1.12 is answering the identical health check more than twenty times slower than its two siblings — a noisy-neighbor process on that specific host, a slower disk, a cold cache that never warmed up, the actual cause varies, but the load balancer has no way to know any of that. Round robin keeps sending this backend exactly one-third of all requests regardless, because "one-third of requests" was the only promise it ever made. Every request that happens to land on 10.0.1.12 takes the full latency hit, and because it's exactly a third of all traffic, the aggregate p99 shows periodic spikes lining up precisely with that rotation.

Plain round robin has no mechanism to react to this on its own — reacting to actual backend performance is a different algorithm's job entirely, covered in the next article in this module.

Practice exercises

  1. A pool of four identical backends handles requests that all cost roughly the same amount of processing time. Explain why round robin is a reasonable choice here, tying the answer to the specific assumption round robin makes.
  2. Using the curl timing check shown above, explain why checking only the load balancer's own aggregate latency metric would have made this scenario's slow backend much harder to find.
  3. A colleague claims that adding a fourth, much more powerful backend to the pool above and leaving plain round robin in place will "automatically balance the extra capacity because it'll just get its fair share of requests." Explain what's wrong with that reasoning.

Round robin's blind spot is specific and correctable: it counts requests instead of watching what a backend is actually doing right now. The next article covers the algorithm built to close exactly that gap.

Sources