Summary

  • RFC 9721 addresses a hidden modeling failure in EVPN-IRB: one sequence number on a combined MAC+IP route cannot independently describe freshness when an IP moves to another MAC, a MAC acquires another IP, or all-active peers learn the same host in different evidence epochs.
  • The RFC retains one wire attribute through parent-MAC sequence authority, child-route inheritance, peer synchronization, maximum-value interoperability logic, ARP/NDP probing and scoped duplicate recovery. A larger number remains a bounded route-version claim—not proof of host identity, physical location, complete convergence or service recovery.

Two Provider Edge devices connect the same all-active Ethernet Segment. A host has just moved from another segment. One PE learns it after the old remote route has disappeared and calls the new local state version zero. Its peer learns the same host while the old route is still present and calls the move N+1.

Both advertisements can be produced by conforming local logic applied at different moments. They name the same host and the same ESI, yet they offer remote PEs two incompatible histories. Selecting the larger number may appear obvious. It does not explain why the numbers diverged, whether the peers synchronized, whether the old binding still answers, or whether either path forwards traffic.

This is the useful difficulty exposed by RFC 9721. Sequence numbers are often treated as if they were small timestamps. In EVPN mobility they are not time. They are protocol evidence created from route state, local learning and comparison rules. They become meaningful only inside the authority that assigned them.

The combined route concealed two independent objects

RFC 7432 defined the MAC Mobility Extended Community. When a MAC moves between Ethernet Segments, a later advertisement can carry a higher sequence number so remote PEs prefer the newer location. That is a disciplined use of a counter: the object being versioned is the MAC route.

EVPN Integrated Routing and Bridging carries more. A Route Type 2 can advertise both a MAC address and an IP address. The MAC enters a bridge table. Depending on the IRB design, the IP-to-MAC association enters ARP/Neighbor Discovery state or the IP route enters an IP-VRF. RFC 9135 describes those forwarding models and the probes used around host movement.

Putting both values in one route does not make them one identity. A virtual machine can keep its IP after a reload but receive a new MAC. A workload can keep its MAC and be reprovisioned with another IP. Several IPs behind a firewall can share one MAC. The first EVPN mobility bargain looks like a one-to-one pair; the operating world supplies one-to-many and many-to-one relationships.

The failure is easiest to see when a fresh pair begins at zero. Suppose IP-a/MAC-a was advertised at sequence 27. The VM is rebuilt as IP-a/MAC-b. If the new pair is treated as an unrelated route and begins at zero, its numerical age contradicts physical reality. Raising only the new pair's number solves the IP comparison but may misstate the history of MAC-b and its other children. The same problem appears in reverse when a MAC retains its identity but its IP changes.

RFC 9721 refuses a tempting redesign. It does not put separate MAC and IP sequence fields into the route. That would complicate the wire format, compatibility and debugging. Instead, it constructs a local dependency: the MAC route becomes the parent version authority, and every associated MAC+IP route inherits the parent's sequence.

Inheritance is a control, not a property of the packet

The inherited number solves a protocol problem only if the implementation preserves the family relationship. When a local MAC+IP binding is learned, the PE computes or recomputes the parent MAC sequence. It must outrank relevant remote state for that MAC. If the IP was previously associated with another remote MAC, the new parent must also outrank that other MAC's sequence. Every local child associated with the parent then inherits the result.

On the wire, a remote observer sees one route and one number. It does not see the local dependency graph that justified the value. The packet does not say which remote records were compared, whether every sibling child was updated, whether a Peer-Sync-Local value was incorporated, or whether a stale child survived in an ARP table.

This is why an audit cannot stop at BGP capture. The useful receipt links the learned MAC/IP pair to its local interface and ESI; the parent MAC state before and after computation; the remote values used as inputs; the list of children that inherited the change; and the exact route advertisements emitted. If one link is missing, a high sequence can be correct in syntax while detached from the state it was meant to order.

The hierarchy also determines failure scope. A duplicate MAC makes its child MAC+IP routes suspect. A duplicate IP bound to another MAC should freeze the affected binding without automatically condemning every other IP sharing the same firewall MAC. Parent and child are connected, but they do not have identical fault domains.

All-active peers need a shared epoch

All-active multihoming turns local correctness into a distributed-state problem. Multiple PEs serve the same Ethernet Segment. A host's frames or ARP/NDP responses may hash to one link before another. Local learning is therefore asynchronous even when the physical attachment is common.

The two-PE opening is not an edge-case curiosity. If one peer advertises zero and another N+1, a remote PE may decline to form equal-cost paths. One local PE can reject a returning remote version as older while its peer accepts it as newer. The segment presents one attachment to the customer but several version authorities to the overlay.

RFC 9721 requires those peers to align local and Peer-Sync-Local MAC and MAC+IP sequence state. A peer-supplied version can raise the local parent; all local children must inherit that raise. The operational object is not merely “the route was synchronized”. It is a shared epoch across every PE entitled to speak for that ESI.

That epoch needs its own evidence. Record peer membership, the synchronized object, the value received, the local value before reconciliation, the winning value, the affected children and an acknowledgement that the new state reached every advertising peer. A green multihoming session or an identical ESI does not prove the sequence database converged.

Taking the maximum restores order, not truth

Mixed implementations complicate the model. A remote PE can advertise one value for a MAC-only route and other values for combined MAC+IP routes with the same MAC. RFC 9721 tells the receiver to interpret the remote MAC version as the maximum of the relevant values and to recompute it when a route is withdrawn. If no MAC-only route exists, the maximum among the combined routes supplies the derived parent version.

This is a sensible compatibility rule. It prevents one smaller sibling from making the receiver forget a larger history already observed. But “maximum” is an ordering operation, not causal discovery. The largest number may have been created by a legitimate move, a peer-sync correction, a race, a stale binding that bounced between PEs, or hostile host-facing traffic.

The distinction matters during incident response. A route collector can establish that value 41 replaced value 40. It cannot establish that a VM moved at a particular wall-clock time, that the new MAC belongs to the intended workload, or that the larger value reached the FIB. A counter is not a clock; a best path is not a physical witness.

The current RFC 9721 errata record sharpens the deployment uncertainty. Rejected Technical Erratum 9001 described a mixed environment in which an RFC 7432-only PE would not know to raise a new MAC's sequence above a value formerly associated with the same IP under another MAC. The Area Director accepted the limitation of that new behavior on a legacy PE but rejected the proposed conclusion of backward incompatibility: the old implementation was never specified to perform the extension, while the existing encoding and earlier behavior still interoperate.

Operators should retain exactly that bounded conclusion. Do not say the erratum corrected the RFC; it did not. Do say that extended mobility behavior depends on the relevant PEs implementing the extension. Version inventory belongs in the evidence chain.

Probing asks whether the old binding still answers

When a PE receives a winning remote MAC or MAC+IP route, route selection alone cannot safely erase local reality. RFC 9721 requires probe-and-delete behavior for related local bindings. RFC 826 supplies the ARP foundation; RFC 4861 supplies IPv6 Neighbor Discovery context. RFC 9135 describes host-mobility probing around EVPN-IRB.

A response to ARP or NDP is valuable running-code evidence: something on the local attachment answered for the address. It still does not authenticate the intended VM, prove the orchestrator's assignment, or show that the remote path works. A non-response may mean the host moved, remained silent, was filtered, suffered loss, or was probed through the wrong local state.

RFC 9721's shared-MAC race shows why the probe is operationally essential. Two IPs can exchange MAC associations across PEs while stale entries remain. Parent-sequence rules can make the new bindings bounce, incrementing values until the old ARP, NDP and MAC entries are removed. More arithmetic does not end the race; retiring the stale observations does.

The closeout record therefore needs the probe target, interface, timing, reply or timeout, table entry deleted, route withdrawn, replacement installed and packet outcome. “Higher route received” is only the trigger.

Duplicate protection can freeze the innocent state

RFC 9161 defines operational Proxy ARP/ND behavior including duplicate IP detection. RFC 9721 applies distinct procedures to three cases: duplicate MACs; the same IP behind different MACs; and IP-only routed-overlay advertisements. RFC 9136 supplies the Route Type 5 context for that routed case.

Move counts within a configured time window can cause an address to be marked duplicate and frozen. That is a protective decision under ambiguity. It is not proof that an attacker spoofed the address. A migration loop, stale synchronization, orchestration conflict or legitimate anycast exception can produce superficially similar evidence.

Recovery begins outside BGP: the wrong host-side assignment must be unprovisioned. Aging may eventually clear the frozen condition. An operator can accelerate recovery by unfreezing the route or clearing the appropriate MAC or ARP/NDP state. Unfreeze then advertises a higher value so peers can relearn.

That command is powerful precisely because it writes a new history. If the wrong endpoint was not removed, unfreeze can restart the conflict with a larger number. If the operator clears the parent too broadly, unrelated children can be disturbed. If only one multihoming peer is repaired, the next synchronization can reintroduce the disputed epoch.

Recovery evidence must therefore join four facts: corrective action at the host; bounded table or route action at the PE; convergence across all authorized peers; and observed packet/application restoration. The sequence increase is not any of those four.

A route-version ledger for real operations

A defensible mobility record should be able to answer thirteen questions without reconstructing them from a dashboard screenshot:

  1. Which orchestrator or operator intended the workload, MAC, IP and attachment to change?
  2. What local frame, ARP or NDP event caused each PE to learn the binding?
  3. What was the parent MAC sequence before and after computation?
  4. Which child bindings inherited it?
  5. Which Peer-Sync-Local values arrived, from whom and in what order?
  6. Which remote advertisements and withdrawals were compared?
  7. Did every relevant implementation support RFC 9721's extended behavior?
  8. How was the remote maximum derived and recomputed?
  9. Which route won, and which RIB, bridge, neighbor and IP-VRF entries changed?
  10. Did the expected FIB entries appear on every forwarding PE?
  11. What did the event-driven probes observe and delete?
  12. Which duplicate threshold, freeze, unprovision, clear or unfreeze action occurred?
  13. Did bidirectional packets and the application complete on the intended workload?

Each question belongs to a different authority. The orchestrator owns intended identity. The access PE owns local learning. The multihoming group owns its synchronized epoch. BGP distributes claims. Route selection chooses among them. The forwarding plane executes a subset. The host and application disclose the outcome.

Lu Heng's minimum-specification doctrine explains why the compact route can remain useful without becoming sovereign. A common wire field coordinates independent implementations; it need not decide workload identity or service success. Reality-layer discipline prevents an advertised number, selected route and functioning application from borrowing one another's authority. Running-code primacy makes synchronized tables, probes, FIB reads and packet outcomes stronger evidence than a tidy control-plane record.

RFC 9721 is therefore not a story about making sequence numbers smarter. It is a story about compensating for what one number cannot know. The protocol preserves interoperability by building disciplined relationships around a limited field. Operations must preserve those relationships as evidence—or the largest number will become the easiest story to believe and the least defensible one to act upon.

Sources