Skip to content

Understanding the client-server model in code

Both servers built in this module so far handle exactly one thing at a time. The TCP echo server calls accept() once and then serves that single connection until it closes — a second client trying to connect during that time would sit in the accept queue, unanswered, exactly as that article's closing section admitted outright. The UDP server at least loops on every datagram, but it still processes them one at a time, in order, on a single thread. Neither is what "client-server" actually means in practice — a real server serves many clients at once, and this article is about the concrete choices that make that possible.

The gap between "accepts one connection" and "is a server"

Go back to the TCP echo server's main() function: accept() is called a single time, outside any loop. The fix for handling more than one client sounds trivial — wrap accept() in while True: — and it is, for the accepting part:

while True:
    conn, addr = server_sock.accept()
    # now what?

The real question is what happens on the line right after. If that loop simply calls the same blocking recv()/sendall() logic inline, right there in the loop body, the server is back to handling one client at a time — it can't call accept() again until the current client's entire conversation finishes, because the same thread is busy blocking on that client's recv(). Making accept() loop is necessary but not sufficient; the connection handling itself has to happen somewhere that doesn't block the loop from reaching the next accept().

Three ways to stop one client from blocking every other one

A new process per connection, using something like Python's multiprocessing or a plain fork() on Linux, gives each client's connection its own completely separate process, with its own memory space. This is the oldest approach — traditional Unix network daemons worked exactly this way — and it has a genuine advantage: one client's connection crashing, or even the process handling it crashing outright, can't corrupt or take down any other client's connection, since they don't share memory at all. The cost is weight: creating a new OS process for every single connection is measurably slower and more memory-hungry than the alternatives below, which matters once connection counts run into the thousands.

A new thread per connection is what the code below actually does, and it's a lighter-weight version of the same idea: each client gets its own thread, sharing the same process and memory space, but running independently enough that one thread blocked on a slow client's recv() doesn't stop the main thread from calling accept() again for the next one.

import socket
import threading

HOST = "127.0.0.1"
PORT = 65435


def handle_client(conn, addr):
    print(f"Connection from {addr}")
    with conn:
        while True:
            data = conn.recv(1024)
            if not data:
                break
            conn.sendall(data)
    print(f"Connection from {addr} closed")


def main():
    server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    server_sock.bind((HOST, PORT))
    server_sock.listen(5)
    print(f"Listening on {HOST}:{PORT}")

    while True:
        conn, addr = server_sock.accept()
        thread = threading.Thread(target=handle_client, args=(conn, addr), daemon=True)
        thread.start()


if __name__ == "__main__":
    main()

The structural change from the original echo server is small but decisive: the per-connection logic — the while True: recv()/sendall() loop — moved into its own function, handle_client(), and every call to accept() immediately hands the new connection off to a fresh thread rather than handling it inline. The main thread's only job now is looping on accept() as fast as connections arrive; it never touches recv() or sendall() itself, so a slow or silent client sitting in handle_client() never prevents the next accept() from happening.

Run this against two clients connecting at roughly the same time, and both get served independently and correctly:

Connection from ('127.0.0.1', 51022)
Connection from ('127.0.0.1', 51024)
Connection from ('127.0.0.1', 51022) closed
Connection from ('127.0.0.1', 51024) closed

Threads genuinely run concurrently from the operating system's point of view, but they share the same process memory — which means a bug in one client's handling that corrupts shared state can affect every other thread, a class of bug ("race conditions") that a separate-process design structurally can't produce, since separate processes share nothing by default.

An event loop, the model behind Node.js and Python's own asyncio, takes a completely different approach: one single thread, and instead of blocking on any one client's recv(), the program registers interest in many sockets at once and lets the operating system report which ones actually have data ready. Nothing blocks waiting for a specific client; the loop simply reacts to whichever socket becomes ready next, in whatever order that happens to be. This avoids the memory overhead of a thread or process per connection entirely, which is exactly why it scales to far higher connection counts on the same hardware — the trade-off is that a single slow, CPU-bound piece of work in the handling code blocks the entire loop, every client, at once, since there's only one thread available to do anything.

None of this is specific to sockets

Every one of these three models — a process per unit of work, a thread per unit of work, an event loop reacting to many things at once — shows up constantly outside networking entirely: a process pool handling background jobs, a thread pool processing image uploads, an event loop driving a UI. Sockets happen to be the place this course introduces the pattern, because "one client, one connection, needs handling without blocking the next one" is the clearest, most concrete version of the underlying problem — but the choice between these three models is a general concurrency decision, not a networking-specific one.

Neither threads nor an event loop are free lunches, they're a trade of one bottleneck for another: a thread costs memory and context-switching overhead per connection; an event loop costs the entire loop blocking if any single handler does something slow. Real production servers frequently combine both — an event loop per worker process, several worker processes running in parallel — precisely to get concurrency within each process and true parallelism across them, which is roughly how Nginx itself is built.

Practice exercises

  1. Using the threaded server above, explain what would happen if handle_client()'s loop had no exit condition at all — no if not data: break — and a client simply stopped sending data without closing its connection. What does the thread do, and does it affect the server's ability to accept new clients?
  2. A colleague suggests removing threading entirely and instead running many separate copies of the entire program, each bound to a different port, to handle more clients. Explain what this approach gains and loses compared to the threaded version, tying the answer to the process-per-connection model described above.
  3. Using the daemon=True argument on the threading.Thread call above, look up what that flag controls, and explain what would happen when the main program tried to exit if it were set to False instead, with client threads still running.

Every server in this module has run inside one program's neatly controlled world: a known port, a trusted client, no firewall in the way, no real network between the two ends beyond loopback. None of that holds once this code runs on an actual machine talking to actual clients over an actual network — and that gap, between code that works in a single process on 127.0.0.1 and a service that survives contact with a real network, is exactly what this course's hands-on module puts to the test next.

Sources