RFC 2260 and the hidden routing cost of multihoming

Summary

RFC 2260 kept provider-allocated enterprise prefixes inside provider aggregates during ordinary operation, then allowed a failed side’s prefix to become a more-specific advertisement through a surviving provider.
Its alternatives did not eliminate multihoming cost: they moved it among temporary global route state, non-direct EBGP and encapsulation, constrained propagation, renumbering, provider coordination and suboptimal paths.
The RFC describes control-plane arrangements and expected properties, not observed deployments or proof that an announcement was accepted, propagated, forwarded or delivered to an application.

The interesting route in RFC 2260 is the one that is absent. In the ordinary state, an enterprise connected to two providers advertises to each provider only the prefix that provider allocated. The provider can carry that customer route inside its larger aggregate. A default-free router elsewhere need not keep a separate entry for the enterprise. Aggregation has preserved a small public surface precisely because the backup route has not yet sent its invoice.

Failure makes the accounting visible. If the border router on the surviving side concludes that reachability through the other provider has disappeared, it begins advertising the failed side’s provider-allocated prefix to its own provider. That more-specific route may then enter the default-free zone for the duration of the outage. When the other path returns, the enterprise withdraws the extra advertisement. RFC 2260 expected simultaneous failures among all multihomed enterprises to be uncommon, so the average extra state would be only a fraction of the population.

That was a design expectation resting on an assumption, not a measurement of routing tables or incidents.

The trigger is therefore as important as the route. In the two-provider example, the router compares the sets of routes reachable through each side. While the intersection is non-empty, the surviving edge advertises only its own provider’s prefix. If the intersection becomes empty, it also advertises the remote provider’s prefix. Because repeated set comparison can be costly, the RFC suggests watching one or more selected backbone prefixes learned through IBGP. Their disappearance becomes a practical signal to inject the backup route.

This mechanism compresses a complicated question—can the other side still carry useful traffic?—into control-plane evidence. It does not turn that evidence into a delivery receipt. A selected backbone prefix can disappear without describing every destination, while its presence cannot prove application delivery. Even after injection, an upstream may reject the route, a prefix-length filter may suppress it, propagation may be partial, or forwarding may fail. The document itself warns that filtering can prevent full Internet-wide connectivity.

Addressing supplies the steady-state foundation. An enterprise attached to N providers receives N prefixes, one from each provider under the address-lending model discussed in RFC 2008. Internal nodes can use the prefix topologically closest to an attachment, or addresses from several prefixes. The chosen address influences which inbound path is natural. It also creates a lifecycle obligation: replacing a provider means renumbering the part of the enterprise that used that provider’s block. The RFC mentions NAT as one possible response to assignment and renumbering questions, but explicitly leaves multihomed NAT outside its scope.

Provider-allocated space is not made portable by the routing design.

RFC 2260 then offers a different place to put the failure cost. An enterprise border router can maintain EBGP not only with its directly connected provider router but also with a router in the provider attached to the other enterprise edge. Directly learned routes are preferred. If the direct link on one side fails, traffic for that side’s prefix can still reach its allocating provider, which encapsulates packets toward the surviving enterprise edge; the enterprise decapsulates them and forwards them internally. RFC 1773 is cited for GRE.

The attraction is architectural: the failure need not create an enterprise-specific route in the default-free zone, and a backup more-specific cannot be defeated by the same prefix-length filtering because it is not being globally injected. But the cost has moved into non-direct peering, authentication, tunnel installation, preference policy and provider cooperation. RFC 2260 says multi-hop EBGP should use suitable peer authentication. It does not demonstrate that providers agreed commercially or operationally, configured the tunnel correctly, accepted every route, or delivered traffic after the failure.

Nor is the tunnel path necessarily elegant. Non-direct EBGP can preserve the small global table while sending packets along a less direct route after failure. The RFC therefore combines it with a modified injection method: distribute an extra route only within a constrained scope, with a BGP Community as one possible tool. A limited optimisation route can improve ingress choice without exposing every default-free router to the change.

The same bargain exists before failure. Advertising only the prefix allocated by the directly attached provider can produce suboptimal paths when a source is a customer of one ISP but the destination node uses another ISP’s prefix. Advertising more provider prefixes can improve those paths. Distributing them too broadly, however, creates additional default-free state. Better path choice can be bought with visibility; stronger aggregation can be bought with address dependence, tunnels, coordination or longer routes.

The comparisons make the cost curve explicit. Provider-independent space gives the enterprise a single prefix independent of its providers, but its enterprise-specific route cannot be folded into a provider aggregate; RFC 2260 characterises the global overhead as O(N) in the number of multihomed enterprises. Taking one provider’s prefix and asking other providers to carry it selectively instead requires proxy aggregation, inter-provider coordination and more complex configuration. Auto-injection exposes a failure and any instability to routers receiving the more-specific, demanding controls for flapping.

Non-direct EBGP localises the churn but adds machinery closer to the enterprise and its providers.

Later RFCs sharpen the context without supplying missing deployment evidence. RFC 2519 explains that aggregation reduces table size, processing and the scope of route flaps, while more-specific information can stay local or carry no-export. RFC 4116 classifies the RFC 2260 approach as provider-aggregatable multihoming with no additional global-table load in its basic form, but notes that it does not meet every goal, notably transport-layer survivability. RFC 8678 later describes provider-assigned IPv6 multihoming as avoiding PI routes while still requiring hosts and routers to coordinate source-address and egress choice. Its later mechanisms should not be projected backward into the 1998 proposal.

RFC 2260 is Informational, published in January 1998 by Tony Bates and Yakov Rekhter. It names reliability, load distribution and potentially better geographic routing as motivations, and says its strategies apply to IPv4 and IPv6. Those statements establish scope and intent, not equal adoption, measured availability, better latency or surviving sessions. The historical lesson is narrower and more useful: every multihoming design chooses where exceptional information appears, who must coordinate it, and how much state remains after the exception ends.

The Backup Route’s Invoice

Why it matters

Summary

Sources

Deeper Profile Context

Strategic Circle

Leadership Alliance

Strategic Circle Briefing

Leadership Alliance Briefing

Summary

Sources

Deeper Profile Context

Strategic Circle

Leadership Alliance

Recommended reading