Skip to content

IP Hash

Every algorithm covered so far in this module — round robin, least connections, and weighting — makes its decision fresh on every single request, with no memory of who asked last time. That's fine for a stateless API where any backend can answer any request identically. It falls apart the moment a backend holds something in memory that only it knows about — the load balancer article's own scenario already walked through exactly this failure: a user's session created on one backend, their next request round-robined to a different one that's never heard of it, and an inexplicable logout. IP hash is one direct answer to that problem — route the same client to the same backend consistently, without the load balancer needing to remember anything about individual sessions at all.

Hashing a client onto a backend

Instead of rotating through backends or comparing their current load, IP hash computes a hash of the client's IP address and uses that hash to pick a backend — the same input address always produces the same hash, which means the same client lands on the same backend every time, as long as the pool itself doesn't change.

NGINX's own documentation for the ip_hash directive is specific about exactly which bits of the address get hashed: "the first three octets of the client IPv4 address, or the entire IPv6 address, are used as a hashing key." For a client at 203.0.113.42, that means 203.0.113 — the fourth octet is deliberately dropped from the calculation. This isn't an oversight; it's a deliberate choice to keep clients on the same /24-sized network segment mapped to the same backend, on the reasoning that many clients behind the same NAT gateway or the same ISP subnet share a common exit address's first three octets and should land together rather than being scattered by a coincidence in their last octet.

Client 203.0.113.42  -> hash(203.0.113) -> Backend B
Client 203.0.113.107 -> hash(203.0.113) -> Backend B   (same /24, same backend)
Client 198.51.100.9  -> hash(198.51.100) -> Backend A

Session affinity can be built at least two ways, and IP hash is specifically the version that needs no cooperation from the client or the application. A load-balancer-issued cookie, the sticky-session approach the load balancer article mentioned as one fix for its session-affinity scenario, requires the client to send that cookie back on every request — which fails outright for a client that doesn't handle cookies at all, a raw TCP connection, or a non-browser client that never stores one. IP hash needs nothing from the client beyond the connection itself; the address is already right there in the IP header on every packet, the same header IP addressing covered back in the first module.

The trade-off runs the other way just as clearly. A cookie identifies one specific client precisely; an IP address, especially the truncated /24-style key NGINX's ip_hash uses, identifies an entire address range that might contain many unrelated clients behind the same corporate NAT gateway or mobile carrier's address pool. Every one of those clients gets pinned to the identical backend under IP hash, which can concentrate far more load onto one server than the hash was ever meant to send its way — a large office or campus network sitting behind a single public IP is the textbook case where this bites.

The rehashing problem: what happens when the pool changes

Here is where IP hash's simplest implementation has a real structural weakness. A naive hash typically works by taking the hash value modulo the number of backends — hash(key) % N — to decide which backend index to use. That arithmetic depends entirely on N, the backend count. Add a fourth backend to a pool of three, and N changes from 3 to 4; the modulo result for nearly every existing client's hash changes along with it, even though only one backend was added and the other three never went anywhere.

3 backends:  hash(client) % 3 -> backend index 0, 1, or 2
4 backends:  hash(client) % 4 -> backend index 0, 1, 2, or 3   (almost all mappings shift)

The practical consequence is that scaling the pool up or down under plain modulo hashing remaps the overwhelming majority of clients to a different backend all at once — precisely the moment session affinity was supposed to prevent that from happening. Every client that had a warm cache entry or an in-memory session on its previously assigned backend loses that locality simultaneously, right when the operator only meant to add capacity.

Consistent hashing is the fix, and it's a different algorithm from plain modulo hashing, not a tuning parameter on top of it. Instead of mapping clients directly onto a small, fixed set of backend indices, consistent hashing maps both backends and clients onto points on a large, fixed-size ring, and assigns each client to the nearest backend point going around that ring. Adding or removing one backend only touches the section of the ring immediately around it — the rest of the ring's mappings hold steady. NGINX documents this precisely for its hash directive's consistent parameter: it "ensures that only a few keys will be remapped to different servers when a server is added to or removed from the group," in direct contrast to the plain hash directive, where the documentation warns that changing the server count "may result in remapping most of the keys to different servers." Google's Maglev paper, describing the software load balancer running in front of Google's own services, cites this exact property — consistent hashing paired with connection tracking — as deliberately built to "minimize the negative impact of unexpected faults and failures on connection-oriented protocols," because a backend disappearing unexpectedly is architecturally the same event as an operator removing one on purpose.

Plain modulo hashing gives session affinity until the pool size changes; consistent hashing gives session affinity that survives the pool changing. That distinction is invisible in a static pool and becomes the entire story the moment autoscaling or a rolling deployment starts adding and removing backends on its own schedule.

Practical scenario: a rolling deploy that logs out half the userbase at once

A team runs a three-backend pool behind NGINX with plain ip_hash, storing session state in each backend's local memory rather than a shared store. A routine deployment adds a fourth backend temporarily to handle the extra capacity during a rolling restart, and support tickets about unexpected logouts spike immediately afterward, clearing up only once the deployment finishes and the pool returns to three backends.

upstream web_pool {
    ip_hash;
    server 10.0.4.10:8080;
    server 10.0.4.11:8080;
    server 10.0.4.12:8080;
}

Reasoning through what ip_hash actually does when a fourth server line is added temporarily, rather than assuming the deployment itself is at fault, explains the timing precisely: going from three backends to four changes the divisor in the hash's underlying modulo arithmetic, which reassigns the large majority of existing clients to a different backend than the one holding their session — exactly the rehashing problem described above, triggered by routine capacity scaling rather than a deliberate backend removal. The moment the temporary fourth backend is removed and the pool returns to three, most clients get rehashed yet again, back toward their original assignments but not guaranteed to land on exactly the same backend they started on, since the hash calculation has no memory of history, only of the current backend count.

The durable fix has two independent parts, and either one alone would have prevented the tickets. Switching from plain ip_hash to NGINX's hash ... consistent mode would have kept the rehashing to a small fraction of clients instead of the majority. Moving session storage out of each backend's local memory and into a shared store like Redis — the same fix the load balancer article's own scenario landed on for its round-robin logout problem — removes the dependency on hitting the same backend at all, which makes the question of which backend a client's request reaches no longer a correctness issue, only a performance one.

Practice exercises

  1. Two clients, 203.0.113.5 and 203.0.113.250, connect to a pool using NGINX's ip_hash. Using the exact hashing key NGINX documents, explain whether these two clients are guaranteed to land on the same backend, and why.
  2. Explain, using the modulo arithmetic shown above, why adding a single backend to a pool of three running plain hash-based routing can remap far more than one-fourth of existing clients, when only one-fourth of the pool's total capacity actually changed.
  3. A team argues that since consistent hashing solves the rehashing problem, there's no remaining reason to also move session storage to a shared store. Using the practical scenario above, explain what gap consistent hashing alone still leaves open.

Every algorithm in this module decides which single backend serves a given request. That decision only matters at all because the load balancer already trusts every backend in its pool to run the identical application and hold, or not hold, session state the same way as its siblings — the algorithm picks among interchangeable choices, not among fundamentally different servers. All four algorithms, and the load balancer running them, ultimately sit on top of one much lower-level building block: the socket a client and a server each open to actually exchange bytes over a connection. That's where this course goes next.

Sources