Summary

  • RFC 5961 accepts an established-connection RST immediately only when its sequence number exactly equals RCV.NXT; an in-window but non-exact reset receives a challenge ACK and is dropped.
  • The same proof pattern protects synchronized connections from blind SYN attacks, while a narrower ACK test can reduce data injection and tunable throttling bounds the new response surface.

The original weakness was not that TCP had no sequence numbers. It was that a destructive command could be accepted across the whole receive window. An off-path attacker did not need to observe the connection; repeated forged RSTs spaced across possible windows could eventually land inside one and tear the connection down. Larger windows reduced the number of guesses.

RFC 5961 changes the authority test. An out-of-window RST is silently discarded. A reset whose sequence number exactly matches the next expected byte, RCV.NXT, still terminates the connection. A reset that is inside the window but does not exactly match no longer wins immediately: the endpoint sends <SEQ=SND.NXT><ACK=RCV.NXT><CTL=ACK>, drops the suspect segment and continues processing the connection normally.

The challenge asks the peer to reveal state

The ACK is not authentication. It is a state challenge. A legitimate remote endpoint that really closed the connection no longer has the corresponding transmission control block. When it receives the ACK, ordinary TCP behavior makes it answer with another RST whose sequence number is derived from the acknowledgement. That second reset now exactly matches and can close the local connection. A blind attacker normally cannot see the challenge and therefore cannot construct the confirming reset.

This can add a round trip to a genuine close, and an out-of-order legitimate RST may first be challenged. The design accepts that delay because the alternative lets a broad probability window carry destructive authority.

The SYN rule uses the same idea in a synchronized state. Instead of letting an acceptable forged SYN trigger a reset, the endpoint sends a challenge ACK regardless of the SYN sequence number, drops the segment and waits for a legitimate restarted peer to confirm closure with a valid RST. In SYN-SENT, the older boundary remains: a RST is acceptable only if its ACK acknowledges the SYN.

RFC 5961 preserves a rare corner case rather than creating it. If a restarted peer reuses the same addresses and ports and happens to choose an initial sequence number exactly RCV.NXT-1, it may ignore the challenge as an acceptable duplicate ACK and keep retransmitting SYN until establishment fails.

Data injection needs two scoped guesses

Blind data injection has a different, optional check. An implementation may accept the incoming acknowledgement only when it falls between SND.UNA-MAX.SND.WND and SND.NXT. MAX.SND.WND records the largest window the local sender has ever received from the peer, including scaling. A segment outside that ACK range is dropped and acknowledged.

The extra dimension makes a blind attacker guess both a usable receive sequence and an ACK within a connection-specific range. It also strengthens resistance to forged FIN segments. It reduces probability; it does not eliminate false injection. The RFC accordingly gives the RST and SYN mitigations SHOULD strength, while the data check is MAY.

The defense creates a resource decision

Every challenge response consumes bandwidth and processing. RFC 5961 therefore recommends tunable ACK throttling. Its example—no more than ten challenge ACKs in five seconds—is empirical, not a universal constant. A tighter limit protects bandwidth and CPU; a looser one can clear legitimate stale state faster. Frequent throttling can itself indicate unusual or hostile conditions.

Middleboxes complicate the boundary. A device that caches or manufactures non-conformant resets may keep provoking challenge ACKs, creating an RST/ACK exchange that throttling must contain. A spoofed in-window RST or SYN can also reflect one ACK toward a victim, but it is not an amplification attack because each induced ACK requires one forged segment.

The sole source is RFC 5961, published on the Standards Track in August 2010. It raises the work factor for off-path guessing but does not authenticate TCP, stop an on-path attacker, or cover forged ICMP and every other disruption vector. The RFC identifies IPsec AH or ESP as the full protection boundary. A challenge ACK means the first segment did not prove authority; it does not prove malice.