MTU
Everything this module has covered so far — the handshake, the sliding window, congestion control — governs how much data may be in flight and how a connection reacts to loss. None of it touches a much more basic constraint: how big is any single piece of data allowed to be in the first place? That limit is set below TCP entirely, at the link layer, and it shapes every decision TCP makes about segment size. It's called the Maximum Transmission Unit, and it's the reason MSS and Fragmentation, the two articles that close out this module, exist as topics at all.
A property of the wire, not of TCP
The MTU is the largest frame a given network interface is willing to send in one piece, measured in bytes, and it belongs to the link layer — Ethernet, Wi-Fi, whatever the physical medium happens to be — not to IP or TCP. It's a hardware and driver property of a specific network interface, and different interfaces along the same end-to-end path can genuinely have different values. A host's own interface reports its configured value directly:
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
mtu 1500 here is the standard Ethernet default, and it's been the de facto value across the public internet for decades, largely because so much intervening equipment assumes it. It isn't a law of physics, though — it's a configured value, and it can be raised or lowered on purpose.
Why 1500, and what changes it
Standard Ethernet's 1500-byte MTU is a historical default rather than a hard physical ceiling, and several environments deliberately use something different. Jumbo frames, typically up to around 9000 bytes, show up inside data centers and on storage networks where every hop is under one organization's control — larger frames mean proportionally less header overhead per byte of actual payload, and fewer frames to process for the same amount of data. That efficiency gain only holds where every device along the path agrees to the larger size; a jumbo frame sent onto a link that doesn't support it doesn't just get handled less efficiently, it either has to be fragmented (see Fragmentation) or fails outright, depending on the flags in play.
The internet at large hasn't converged on jumbo frames precisely because that agreement can't be assumed. A packet crossing from a data center's internal jumbo-frame network out onto the public internet will, sooner or later, hit ordinary 1500-byte Ethernet somewhere along the path, and the smaller value has to be respected from that point on. This is exactly the situation the next section exists to handle.
When two hosts disagree: Path MTU Discovery
A connection's two endpoints can each have a different configured MTU, and neither one, by itself, knows what the smallest MTU anywhere along the full path between them actually is — that smallest value, called the Path MTU, is what actually limits how large a single IP packet can be for that specific connection without needing to be split up somewhere in the middle.
Path MTU Discovery (PMTUD) is the mechanism that finds this out, and it works by deliberately provoking a failure rather than guessing. A sender transmits a packet at its own interface's MTU with the Don't Fragment (DF) flag set in the IP header — an explicit instruction that if this packet is too large for some link along the path, it must not be split up, it must simply fail. If a router partway through the path has a smaller MTU on its outbound interface than the packet's size, it can't forward the packet as-is and, because of the DF flag, can't fragment it either. Instead, it drops the packet and sends back an ICMP "Destination Unreachable / Fragmentation Needed" message, which includes the MTU of the link that couldn't accept the oversized packet. The sender receives this, lowers its assumed path MTU accordingly, and retries with a smaller packet — repeating the process, if necessary, until a size gets through cleanly.
Host A (MTU 1500) Router (MTU 1400) Host B
| | |
| --- packet, 1500 bytes, DF ----->| |
| X (too big, DF set,
| can't fragment)
| <---- ICMP: Frag needed, |
| MTU=1400 -----------------|
| |
| --- packet, 1400 bytes, DF ----->| -------------------->|
| | (fits, forwarded) |
This exchange depends entirely on that ICMP message actually making it back to the sender — and that dependency is exactly where PMTUD's most notorious failure mode comes from.
When PMTUD silently fails: the black hole
Plenty of firewalls, on the reasonable-sounding theory that ICMP is mostly useful to attackers doing reconnaissance, block ICMP traffic outright, including the specific "Fragmentation Needed" message PMTUD depends on. When that happens, a router that needs to shrink a packet still drops it — the DF flag hasn't changed — but the sender never receives the message telling it why, or what size would actually work. From the sender's perspective, it just... stops getting a response, for packets over a certain size, on a certain path, while smaller packets and short exchanges keep working perfectly normally. This specific, hard-to-diagnose failure mode has an established name: a PMTUD black hole, and it produces one of the more frustrating patterns in real-world network troubleshooting — a connection that establishes fine, exchanges small amounts of data fine, and then stalls indefinitely the moment either side tries to send something larger than the path's true, undiscoverable-because-blocked-ICMP MTU.
Practical scenario: an API that works for small requests and hangs for large ones
A partner integration calling an internal API reports a strange, consistent pattern: small requests (an authentication check, a status lookup) complete instantly and reliably. Any request with a larger payload — an upload of a few hundred kilobytes — hangs indefinitely and eventually times out, on this one specific partner's network, while every other client integrates without issue.
The TCP handshake itself completes normally in every case — this isn't a connectivity or firewall-blocking-the-port problem, since the connection reaches ESTABLISHED and the small requests genuinely complete. A capture on the server during a hung large request shows data going out normally at first, then a segment sent and never acknowledged, followed by repeated retransmissions of that same segment with no response at all — not a RST, not an ICMP error, nothing:
13:01:02.100010 IP 10.0.5.15.443 > 198.51.100.20.51890: Flags [P.], seq 1:1461, ack 200, win 700, length 1460
13:01:02.400220 IP 10.0.5.15.443 > 198.51.100.20.51890: Flags [P.], seq 1:1461, ack 200, win 700, length 1460
13:01:03.001880 IP 10.0.5.15.443 > 198.51.100.20.51890: Flags [P.], seq 1:1461, ack 200, win 700, length 1460
The same 1460-byte segment retransmitted repeatedly with no reply at all — no reset, no ICMP error, nothing — matches a PMTUD black hole precisely: some link on the path to this specific partner has a smaller MTU than 1500, an intermediate device is silently dropping the oversized packet, and either that device or a firewall further along the path is swallowing the ICMP "Fragmentation Needed" message that would normally tell the server to shrink its packets. Small requests never triggered the problem because they never produced a full-sized 1460-byte segment in the first place.
Lowering MTU or MSS affects every connection on the interface, not just the affected partner
The two common fixes — lowering the server's own MTU, or (more surgically) clamping the MSS specifically for traffic on the affected path — both change how packets are sized for potentially every connection through the interface involved, not just the one partner having trouble. Confirm the current MSS clamping and MTU configuration first (ip link show, and any existing iptables/nftables MSS-clamp rules), test a change against a copy of the partner's actual path characteristics if at all possible, and monitor other traffic through the same interface afterward rather than assuming a fix scoped to one symptom couldn't affect anything else.
The most common actual fix doesn't touch MTU directly at all — it clamps the MSS (covered next, in MSS) for outbound traffic to a safely smaller value, which avoids ever generating a packet large enough to trigger this specific black hole in the first place, sidestepping the broken ICMP path entirely rather than depending on it to work.
Practice exercises
- Explain why raising a data center's internal MTU to 9000 bytes (jumbo frames) is a reasonable, common optimization for east-west traffic between servers in the same facility, but not something you'd expect to help — or even safely attempt — for a connection going out to the public internet.
- Using the PMTUD exchange diagram in this article, explain precisely which single change in network configuration (not in either endpoint's code) would turn the working PMTUD example into the silent black-hole failure described afterward.
- A support engineer says "PMTUD failed, so the connection should just fail cleanly with an error." Using the black-hole scenario above, explain why the actual failure mode is a hang or timeout instead — what message, specifically, never arrives to tell the sender what went wrong?
- Explain why the API scenario in this article affected only large-payload requests from one specific partner, while every other client and every small request worked normally — connect this back to what actually determines whether a given packet triggers the black hole at all.
MTU sets the ceiling for an entire IP packet, headers included. MSS is the number that actually matters to TCP directly: how much of that ceiling is left over for real application data once the IP and TCP headers have taken their share.