Summary

  • A BGP End-of-RIB marker reports completion of the initial routing update for one address family after session establishment. It is not a global freshness certificate for every AFI/SAFI.
  • Graceful Restart deliberately permits a receiving speaker to retain routes and mark them stale while forwarding continues. Route replacement, stale removal, local selection, FIB programming and data-plane viability remain separate evidence.
  • Operators need a per-peer, per-family restart-to-freshness receipt that binds restart cause and negotiated capability to retained routes, EOR, stale cleanup, local forwarding state and live path tests.

The session recovered before the evidence did

Consider a router that has re-established a BGP session after a restart. The IPv4-unicast End-of-RIB marker arrives. A monitoring system interprets that event as convergence and changes the session to green. Yet a second address family still contains routes retained from the old session, and traffic for one of those routes is following a forwarding entry built before the restart.

Nothing in the first EOR contradicts that state. The marker closes the initial update for a particular address family. It does not speak for every family negotiated on the session, and it does not itself test whether preserved forwarding remains viable. The operational mistake is not trusting EOR; it is expanding a scoped protocol fact into a network-wide freshness verdict.

That expansion is especially tempting during Graceful Restart because continuity is the feature’s purpose. Retaining forwarding state can prevent unnecessary churn and transient loss. But the very mechanism that protects continuity also creates an interval in which “still forwarding” and “freshly validated” are not the same condition.

What EOR actually closes

RFC 4724 defines EOR as an UPDATE with no reachable or withdrawn NLRI for IPv4 unicast. For other address families, the marker uses MP_UNREACH_NLRI with no withdrawn routes for the particular <AFI, SAFI>. A speaker sends it after completing its initial routing update for that address family, including when there is no update to send.

The scope is therefore built into the encoding. An IPv4-unicast EOR says nothing about IPv6 unicast, VPN routes, EVPN or any other separately negotiated family. Even within the family, the marker indicates that the sender has completed the update it intends to send; the receiver must still apply those updates, replace stale routes, remove any that remain stale, run local selection as required and program forwarding.

RFC 4724 also separates EOR generation from the ability to preserve forwarding. A speaker may advertise the Graceful Restart Capability without listing any address family simply to indicate that it will generate EOR and support a restarting peer. Seeing the capability or the marker cannot therefore be treated as proof that this speaker preserved usable forwarding state.

The Restart State bit solves a different coordination problem. A restarting speaker sets it in the new OPEN, and a peer must not wait for EOR from a speaker carrying that bit before advertising routes back to it. Once the session returns, the restarting speaker defers route selection for an address family until it has received EOR from the relevant peers or its Selection_Deferral_Timer expires. Only then does it select routes, update forwarding, remove its own stale information and send its EOR for that family.

An EOR timestamp without this preceding sequence loses the distinction between peer-update completion and local forwarding completion.

Retained routes are explicitly stale

When a receiving speaker detects the loss of a session from a peer that advertised Graceful Restart for an address family, it retains the peer’s routes for that family and marks them stale. During forwarding, it does not distinguish those stale routes from other routing information. This is intentional continuity, not an assertion of freshness.

The cleanup rules provide the missing clock. If the session is not re-established within the peer’s advertised Restart Time, retained stale routes must be removed. After re-establishment, they must also be removed immediately for a family if the new capability does not include that family, its Forwarding State bit is not set, or the capability is absent. New updates replace stale routes. When EOR arrives for that family, any routes from the peer that are still marked stale must be removed.

These transitions matter more than the colour of the session tile. Before EOR, a retained route may still be waiting for replacement. At EOR, the receiver gains a boundary for removing the remainder. After EOR, local route selection and forwarding updates still have to yield a usable outcome. An incident record that stores only “EOR received” discards the identities and fate of the routes that made the event operationally important.

Forwarding viability is a different observation

The per-family Forwarding State bit concerns preservation of forwarding state during the restart. RFC 4724 ties it to actual preservation, with a configuration qualification. It is not a packet test. The RFC explicitly leaves the mechanism for deciding whether a peer’s forwarding remains viable outside its scope, mentioning BFD and layer-two monitoring only as examples.

That boundary is decisive. A preserved next hop can become unusable while routes remain retained. The local FIB can lag the RIB transition. Traffic can blackhole or loop even though the session has returned and an EOR has been processed. Conversely, a temporary stale designation does not prove that packets are failing; retained forwarding may be doing exactly what Graceful Restart was designed to preserve.

RFC 8538 extends graceful handling to eligible NOTIFICATION messages and Hold Time expiry when peers exchange the Graceful Notification N bit. In that case both speakers act as receiving speakers and retain one another’s routes. A Hard Reset instead requests a full reset. The extension makes the stale timer mandatory, suggests 180 seconds as a default, and warns that long or infinite retention can create operational problems. It also relaxes one deletion rule for consecutive restarts when the N bit was exchanged.

Those extensions widen the cases in which retained state can be present and make the timer and viability record more important, not less.

Sources