Summary

  • RFCs 2403 and 2404 both put the first 96 bits of a computed HMAC into ESP or AH, even though their full outputs and required keys differed.
  • Later ESP/AH guidance separated the algorithms: RFC 8221 lists HMAC-MD5-96 as MUST NOT and HMAC-SHA1-96 as MUST-. The shared tag length never guaranteed a shared security judgment.

One field, two constructions

In November 1998, RFCs 2403 and 2404 specified keyed authentication transforms for IPsec's Encapsulating Security Payload (ESP) and Authentication Header (AH). One paired HMAC with MD5; the other paired HMAC with SHA-1. Both were designed to provide data-origin authentication and integrity when the secret key was restricted to the communicating parties. Neither transform supplied confidentiality by itself.

The packet-facing number looked the same: 96 bits. RFC 2403's full HMAC-MD5 result is 128 bits, while RFC 2404's full HMAC-SHA-1 result is 160 bits. For either transform, a sender stores the first 96 bits in the authenticator field. A receiver computes the full HMAC and compares those first 96 bits. The 96-bit choice matched AH's default authenticator length and gave the two different constructions a common wire footprint. It did not turn either complete digest into a 96-bit hash.

Their key rules differed too. RFC 2403 requires a 128-bit HMAC key; RFC 2404 requires 160 bits. Thus the same field length coexisted with different full output lengths and fixed key lengths. Negotiated transform identity and correct key handling still mattered; counting visible tag bits alone could not describe the whole security mechanism.

Collision resistance is not the whole HMAC question

The 1998 documents did not equate a collision in an unkeyed hash with an immediate break of keyed HMAC. RFC 2403 said HMAC relied on strong collision resistance to a lesser degree than MD5-based signatures and reported no practical attack on HMAC-MD5-96 at that time. RFC 2404 made a parallel period-specific assessment for HMAC-SHA-1-96. These were dated analyses, not guarantees for future use.

The implementation guidance shows how the evaluation changed without making tag length the deciding variable. RFC 4305 in 2005 put HMAC-SHA1-96 at MUST and HMAC-MD5-96 at MAY. RFC 4835 in 2007 retained that split and noted that the then-known collision weaknesses should not affect either hash with HMAC. RFC 7321 in 2014 acknowledged theoretical results against HMAC-MD5 but said there was no apparent practical vulnerability and that removing it from existing protocols was not urgent; it continued to regard HMAC-SHA-1 as secure despite SHA-1 collision weakness.

RFC 8221, published in 2017, drew a sharper line for ESP and AH implementation requirements: HMAC-MD5-96 became MUST NOT; HMAC-SHA1-96 moved from MUST to MUST-. It cited MD5's known collision vulnerability for the prohibition, while attributing the SHA-1 downgrade to an industry-wide trend to deprecate its use. The latter was not a claim that the 96-bit field had changed or that RFC 2404 had suddenly specified a different tag.

A status table is not a traffic trace

RFC 9395 later updated RFC 8221 and changed an IKEv2 transform registry, including a DEPRECATED entry named AUTH_HMAC_MD5_96. That registry is not the same thing as RFC 8221's ESP/AH implementation table. The identical-looking name crosses adjacent IPsec contexts, so a registry label must be read with its protocol and document scope attached.

These documents record protocol parameters and dated implementation recommendations. They do not count deployed devices, identify the last negotiated HMAC-MD5 security association, or establish a universal retirement date. For an operator, the practical record would need to distinguish the selected protocol, transform identifier, peer capabilities, installed association and observed traffic. A 96-bit tag is only one part of that evidence.

Sources