Skip to content

TCP for file transfer and APIs

Where the previous article looked at applications happy to trade correctness for speed, this one is about the opposite: applications where a wrong or missing byte is a bug, not an acceptable glitch. A file that arrives with a corrupted chunk isn't a "mostly fine" file. An API response missing half its JSON isn't a "pretty good" response. A database write that lands out of order relative to another write on the same row is a data integrity incident.

A wrong or missing byte is a bug, not a glitch. That's the one-line reason file transfer, APIs, and databases all run on TCP: its handshake, retransmission, and ordering aren't overhead to tolerate — they're the entire reason the software works at all.

Why file transfer needs every guarantee TCP makes

Copying a file across a network sounds simple, but think about what has to be true for the copy to be usable: every byte present, none corrupted, all in the original order, and a clear signal when the transfer is actually finished. UDP provides none of that by default — you'd need to reimplement all four properties yourself, badly, on top of a protocol that was deliberately built without them.

FTP, SFTP, and HTTP-based downloads all run over TCP for exactly this reason. Consider what a single dropped segment would mean for a binary file if nothing retransmitted it: not a visible glitch like a skipped video frame, but a silent gap in the file's byte stream — potentially a corrupted archive, a binary that won't execute, or a document that won't open. TCP's retransmission means the application layer never even sees that a segment was lost; the transport layer quietly resends it and the file-transfer protocol above never has to reason about network conditions at all. That's the real value of TCP's guarantees: they let application authors write file-transfer logic as if the network were perfectly reliable, because from their vantage point, it is.

Large transfers also benefit from TCP's congestion control in a way that's easy to overlook. A naive UDP-based bulk transfer with no congestion awareness would blast data at line rate regardless of what the network path could actually sustain, causing exactly the kind of congestion collapse that TCP's slow start and congestion avoidance mechanisms exist to prevent (covered in depth later in this module). File transfer isn't just "needs reliability" — it's a genuinely bulk, sustained workload, which is precisely the case TCP's flow and congestion control were designed around from the start.

Why request-response APIs need ordering and reliability

An HTTP API call is a short-lived exchange with a strict expectation: the client sends a request, and it gets back exactly the response that request produced, complete and intact. Losing part of a JSON response isn't a smaller, mostly-usable response — it's an unparseable one. A client that receives {"status": "ok", "balance": 4 and nothing more doesn't have partial information; it has a parse error. TCP's guarantee that a stream of bytes arrives complete and in order is what lets a client library and a JSON parser trust the byte stream at all.

This is also why REST and GraphQL APIs, gRPC, and virtually every backend-to-backend protocol in common use are built on HTTP over TCP (HTTP/3 is the interesting exception, and it earns a full explanation later in this course rather than a rushed mention here). A modern HTTP client typically keeps a TCP connection open across multiple requests — HTTP keep-alive — specifically to amortize the cost of TCP's handshake across many calls instead of paying it on every single one. This is the direct counterpart to the DNS case from the previous article: DNS pays UDP's near-zero setup cost on every isolated query, while an API client pays TCP's setup cost once and then reuses the connection, because the two workloads sit at opposite ends of "one-shot" versus "conversational."

Why database connections are TCP almost without exception

A database connection carries something that would be genuinely dangerous to lose or reorder: transactional statements, where the order of operations directly determines the result. If a client sends an UPDATE followed by a COMMIT, and the network reordered them, or the COMMIT arrived while the UPDATE was still missing, the outcome would be silent data corruption, not a visible error. TCP's in-order, reliable delivery is what makes it safe for a driver to assume that whatever it wrote, in whatever order it wrote it, is what the server actually processes — a client library that had to defend against out-of-order SQL statements arriving over an unordered transport would be significantly more complex, and there would be no way to make it fully safe.

This is also why running a database connection through certain kinds of tunnels is riskier than it looks. Tunneling one TCP stream (the actual database connection) inside another TCP stream (a TCP-based VPN, for instance) can produce interacting retransmission and congestion-control loops between the two layers — each session reacting independently to the same underlying packet loss — a problem the previous article touched on for VPNs generally. It's a good reminder that TCP's guarantees are strongest when there's exactly one TCP layer between the application and the network, not two stacked on top of each other.

A practical scenario: an API that's fast on paper but slow under one condition

An internal API works fine in every environment except one: clients connecting from a satellite office with a genuinely high round-trip time to the data center — around 220 ms, versus 15 ms for every other office. Requests that should complete in well under 100 ms are consistently taking 400–600 ms, and the API server's own processing time, checked in its logs, is a few milliseconds. The slowdown is happening somewhere in the network path, not in the application.

A packet capture from a client machine at the satellite office, taken while making one request, shows something specific:

sudo tcpdump -i eth0 -n host api.internal.example and port 443 -c 8

09:12:01.001100 IP 10.20.4.15.53211 > 10.1.0.50.443: Flags [S], seq 100200300
09:12:01.221430 IP 10.1.0.50.443 > 10.20.4.15.53211: Flags [S.], seq 900800700, ack 100200301
09:12:01.221900 IP 10.20.4.15.53211 > 10.1.0.50.443: Flags [.], ack 900800701
09:12:01.222400 IP 10.20.4.15.53211 > 10.1.0.50.443: Flags [P.], seq 100200301:100200410
09:12:01.443210 IP 10.1.0.50.443 > 10.20.4.15.53211: Flags [.], ack 100200410
09:12:01.663550 IP 10.1.0.50.443 > 10.20.4.15.53211: Flags [P.], seq 900800701:900801850
09:12:01.884001 IP 10.20.4.15.53211 > 10.1.0.50.443: Flags [.], ack 900801850

The timestamps tell the story before the payload does. From the client's SYN at 09:12:01.001100 to the SYN-ACK at 09:12:01.221430 is 220 ms — one full round trip, as expected for the handshake. The client's HTTP request goes out at .222400, and the server's acknowledgment doesn't arrive until .443210 — another 220 ms, just to confirm receipt, before the server's actual response data appears at .663550, itself another 220 ms later. Three separate round trips are visible here on top of whatever the server actually took to compute the response — the handshake (one round trip), and then a further gap between the request being acknowledged and the response payload actually being sent.

That gap between the ACK at .443210 and the response payload at .663550 is the detail worth chasing. This pattern — an immediate ACK for the request, then a distinct delay before the response body follows — is often Nagle's algorithm interacting badly with delayed ACKs: Nagle's algorithm holds small outgoing segments briefly, hoping to bundle them with more data rather than send many tiny segments, while the receiving side's delayed-ACK logic holds its acknowledgment briefly, hoping to piggyback it on outgoing data of its own. When both sides do this to each other at once, each is waiting for the other to send something, and the deadlock only resolves when a timer on one side finally expires — commonly adding a fixed delay in the range of 40 ms, though the specific value is a kernel-configurable timeout, not a network property, and it compounds badly precisely when round-trip time is already high, because each stall is one more multiple of an already-large baseline delay.

The practical fix, once this pattern is confirmed, is disabling Nagle's algorithm for latency-sensitive connections — setting the TCP_NODELAY socket option — since Nagle's coalescing benefit matters far more for high-frequency, small writes on a low-latency network than for infrequent request-response calls where every held-back segment is now costing a full extra delay on an already slow path. This is a client-and-server-side application setting, not a firewall or routing change, and it's worth confirming with a second capture afterward that the gap between ACK and response payload has actually closed, rather than assuming the fix worked.

Distance is the one variable this fix can't touch

Disabling Nagle's algorithm removes an artificial stall; it does not shrink the 220 ms the packets spend traveling to the satellite office and back. Even after the fix, that office's baseline API latency will remain higher than every other office's, because the underlying constraint is physical distance and the finite speed of light through fiber — not a protocol misconfiguration. Set expectations and timeouts accordingly rather than treating the post-fix latency as another bug to chase.

Practice exercises

  1. Using the packet capture above, calculate how many total round trips (handshake included) this one request took, and how much of the observed 400–600 ms delay each round trip actually accounts for.
  2. A teammate suggests moving an internal reporting API from TCP-based HTTP to a custom UDP protocol "to cut latency." Using this article's reasoning about JSON parsing and partial responses, explain what would have to be built at the application layer to make that safe — and why most teams find that not worth it.
  3. Explain why a database driver can safely assume that two SQL statements sent back-to-back on the same TCP connection will be processed by the server in the order they were sent, and why that assumption would not hold if the same two statements were sent as two separate UDP datagrams.

Sources