Summary

  • A SYN flood exploits a timing asymmetry: a TCP listener commits finite half-open state before the initiator has completed the handshake, allowing cheap requests to occupy the server's backlog.
  • RFC 4987's strongest endpoint defenses either keep a smaller bounded SYN cache or encode recoverable state in a SYN cookie, delaying the full commitment without pretending that the tradeoffs disappear.

A TCP listener faces a small decision with system-wide consequences. A SYN arrives, and the server can remember enough to continue the handshake. That is polite under ordinary traffic. Under hostile traffic it is also a promise: memory and timer work have been assigned to a peer that has not yet shown it can receive the SYN-ACK.

RFC 4987 describes the resulting asymmetry. Conventional TCP retains state in SYN-RECEIVED while it waits for the final ACK. The half-open backlog is finite. An attacker sends enough SYNs, often from addresses that do not answer, and legitimate attempts find no room. The RFC's subject is denial of service against host TCP state and the listening application. A flood that overwhelms the network link itself is a different problem and remains outside this mechanism's protective boundary.

The attack need not have one canonical shape. Spoofed addresses make replies disappear, but compromised machines can send from usable addresses too. The essential ingredients are more basic: the responder allocates state for each opening request, the state pool is limited, and the requests arrive fast enough that allocation outruns reclamation. In the case RFC 4987 analyzes, the immediate damage falls on new inbound connections to the targeted listening port rather than on connections already established.

Several intuitive repairs merely renegotiate the disadvantage. A larger backlog gives the defender more memory but gives the attacker a larger number to fill. A shorter SYN-RECEIVED timer releases entries sooner but can discard legitimate handshakes delayed by ordinary networks; an attacker can answer by increasing the arrival rate. Recycling the oldest half-open control block makes age a victim-selection rule without proving which opener is false. Ingress filtering is valuable when spoofing is the dependency, but it cannot be assumed everywhere and does not neutralize compromised hosts using routable addresses.

Remember less, but remember deliberately

A SYN cache changes what the listener buys at the first knock. Instead of allocating a full TCP control block, it stores a smaller record sufficient to complete the handshake later. The cache is bounded, and RFC 4987 describes distributing entries through a secret-keyed hash. That detail matters: a public bucket function would let an attacker concentrate requests on one bucket even when the total cache still had capacity. Secret distribution makes targeted concentration harder while explicit limits bound memory and lookup work.

The cache is reduced state, not no state. It must still manage expiry and should preserve the server's ability to retransmit a lost SYN-ACK. A design that emits one reply and forgets the retransmission responsibility would convert ordinary loss into connection failure. The operational virtue of the cache is therefore restraint: it postpones the expensive object without erasing the transport behavior needed before the final ACK.

Put the receipt inside the reply

SYN cookies pursue the stronger version of the same principle. The server retains no SYN-RECEIVED record for the opener. It constructs its initial sequence number so that a later ACK can carry back enough evidence to validate the exchange and reconstruct the connection state. Only after that valid completing ACK arrives does the server allocate the full control block.

Appendix A of RFC 4987 illustrates the ingredients: the initiator's sequence number, an encoding of the negotiated maximum segment size, a time counter, the address and port pair, and a secret function. When the ACK returns, the server checks the recovered value against recent counter values and its secret. This is a compact receipt for one handshake, not authentication of a person or application. It shows that the sender of the ACK possessed information from the SYN-ACK, subject to the construction's guessing resistance; it does not establish durable identity.

Statelessness also has a price. State normally retained between SYN and ACK can include negotiated TCP options. A cookie has limited space to encode them, so common constructions can constrain window scaling, selective acknowledgements or future option state. The RFC also calls out data carried on the SYN and a subtler retransmission problem: if the final ACK is lost and an application waits to speak until it receives client data, the server may lack state from which to retransmit the SYN-ACK. The resource promise was delayed, but some protocol memory was delayed with it.

The choice need not be absolute. A listener can use a SYN cache under normal pressure and switch to cookies when that bounded cache fills. The hybrid preserves richer negotiation while capacity exists and reserves the stateless receipt for overload. Firewalls and proxies can perform related mediation, but they move the state burden and may alter end-to-end semantics. They do not repeal the resource decision; they relocate it.

RFC 4987 is an Informational document published in August 2007, not a Standards Track mandate. Its analysis regards SYN caches and SYN cookies as the viable endpoint modifications of its time, while treating the cache as the easier default where its tradeoffs are acceptable. That is a historical recommendation, not evidence of what any current operating system enables.

The durable contribution is a control rule. Do not make an expensive, scarce and attacker-directed commitment at the first unverified message if a cheaper reversible representation can carry the exchange forward. Delay the full state until the initiator returns evidence from the reply. The evidence is imperfect and the deferral can discard useful context, so the mechanism must expose both the resources it saves and the semantics it weakens.

The sole source is RFC 4987, “TCP SYN Flooding Attacks and Common Mitigations”. Its claims here are kept to TCP endpoint state, the defenses the RFC evaluates and their stated limits; it does not establish current defaults, deployment share or attack frequency.