Summary

  • TCP MSS is a SYN-only announcement of the largest TCP data payload its sender can receive in one segment; each direction can carry a different value.
  • RFC 6691 clarified that receivers should advertise reassembly capacity without subtracting option space, while senders account for actual headers when constructing each packet.

Two SYNs, two receiving boundaries

The Maximum Segment Size option looks like a tiny negotiation. It appears while a connection is being established, it carries a number, and the peer is expected to obey it. Yet the word “negotiation” gives the exchange an authority it never had. When host A sends an MSS option, A is describing A's receive side. When host B replies with its own option, B is describing B's receive side. The values may differ without disagreement because they govern opposite flows.

The wire format has barely enough room for ambiguity: Kind 2, Length 4, then a 16-bit value. RFC 9293 defines that value as the maximum receive segment size at the endpoint that sends the option. It may appear in an initial connection request with SYN set and must not appear in ordinary later segments. The option therefore installs a constraint at the moment the endpoints create connection state; it is not a stream of revised capacity reports.

What the number measures matters. MSS counts TCP data octets, excluding the TCP and IP headers. It is not an IP datagram size, an interface MTU, an application write size, or a promise that the path can carry a packet of some total length. A sender uses the peer's announcement as one limit on payload. It must still account for the route and the headers it actually places around that payload.

This division of labour was already central to RFC 879 in 1983. The document called the MSS announcement “often mistakenly called a negotiation” and described it as a statement from data receiver to data sender: I can accept TCP segments carrying up to this much data. The sentence preserved directionality at a time when hosts, gateways, reassembly limits, and variable header lengths made a single universal packet number especially tempting—and especially misleading.

Why 536 became a default, not a law of the path

RFC 879 worked through the default IPv4 arithmetic. Every host was required to reassemble a 576-octet IPv4 datagram. With the minimum 20-octet IPv4 header and minimum 20-octet TCP header, 536 octets remained for TCP data. That relationship supplied the familiar default MSS for the case where no option was sent.

The arithmetic was a compatibility floor built from specified minima. It did not establish that every real path used a 576-octet MTU, that Ethernet fixed a universal answer, or that a receiver announcing 536 had measured anything between itself and its peer. The receiver described what it could accept under the protocol's default assumptions. The sender still possessed local knowledge about the first hop and might acquire other path information through mechanisms outside the MSS option.

The historical discussion exposed a layering problem. IP controlled datagram size and fragmentation; TCP controlled segment payload; both protocols could have options that changed header length. If TCP treated an MSS as a complete packet budget, it would mix a receive capability with assumptions about headers and routes that belonged elsewhere. If IP and TCP shared no information at all, the sender could construct packets that exceeded a known limit. The solution was not to grant one layer omniscience. It was to define what each number meant and require the sender to combine constraints at transmission time.

That distinction also keeps MSS separate from path MTU discovery. PMTUD tries to learn a property of a route and can change as the route changes. MSS is exchanged with an endpoint at connection opening and describes that endpoint's receiving limit. A sender may choose a smaller payload because of path information, but the smaller choice does not prove that the receiver's advertised capacity changed.

The correction about options

Variable headers produced a persistent implementation error. Suppose a receiver can reassemble an IP datagram containing a certain amount of TCP data when minimum headers are used. It might be tempted to subtract anticipated TCP timestamps, IP options, or other header space before advertising its MSS. That seems conservative, but it places packet-construction accounting in the wrong half of the exchange.

RFC 6691 clarified the rule. The receiver should announce the largest TCP payload it can reassemble, measured under the definition of MSS. It should not reduce that advertisement merely because packets might later contain IP or TCP options. When the sender constructs an actual packet, the sender reduces that packet's data as necessary to accommodate the headers that are actually present within the applicable IP maximum.

The timing of the two decisions is different. The receiver makes a durable statement during the SYN exchange about its payload capacity. The sender makes repeated packet-level decisions using the peer's statement, current header lengths, and its own size constraints. Reserving hypothetical option space in the advertised MSS would bake one anticipated packet shape into a connection-wide receive limit and could unnecessarily reduce payload even when those options were absent.

Nor does a large offer allocate resources by itself. The endpoint's implementation and buffers determine what it can safely announce. The peer remains constrained by congestion control, path limits, and its own sending policy. An MSS of 8,000 does not grant permission to send 8,000 data octets on a path that only supports smaller packets, and an MSS of 1,460 does not certify that every hop accepts a 1,500-octet IPv4 packet.

A boundary that middleboxes can rewrite but not discover

Operational networks sometimes alter MSS values in SYN packets, for example to accommodate tunnels or avoid oversized packets on paths where discovery is unreliable. The mechanism may be useful in a particular deployment, but its semantics remain exact: rewriting the option changes the receive constraint presented to the peer. It does not transform MSS into an authenticated measurement of the path.

That matters for diagnosis. Seeing an MSS value in a trace proves that a value was carried at the observation point. It may reflect the endpoint's original announcement or an intermediary's rewrite. It does not reveal every downstream MTU, show whether later packets used the full allowance, or prove why a smaller segment was chosen. The evidence is local to the packet, direction, connection opening, and observation point.

The current TCP specification includes MSS in the mandatory option set. That continuity is not evidence that the Internet converged on one segment size. Read across the three RFCs, it instead shows a stable vocabulary for asymmetric limits and layered knowledge.

The historical lesson is therefore less about one famous number than about disciplined authority. The receiver can state what it can accept. The sender can decide what it can safely emit after combining that statement with its own path and header knowledge. Neither side, and no four-byte option between them, can make the path promise more than it knows.

Sources