Summary

  • Anja Feldmann and five collaborators modeled backbone demand as traffic entering at one link and heading towards a set of possible exits, separating what users wanted to send from the route chosen by the current configuration.
  • Their AT&T study joined NetFlow, forwarding tables, router configurations and SNMP checks; it preserved loss, stale snapshots and ambiguous ingress points instead of presenting a link counter as a complete traffic matrix.

Begin with one heavily loaded backbone link. Its byte counter is accurate enough to warn an operator, but it cannot name the demands that produced the load. It cannot distinguish a large customer flow from hundreds of smaller ones. Most importantly, it cannot answer a counterfactual: if an OSPF weight changes or a BGP route moves, which traffic will leave this link, and which other link will receive it?

That gap shaped a line of work led jointly by Anja Feldmann, Albert Greenberg, Carsten Lund, Nick Reingold, Jennifer Rexford and Fred True. In their 2001 paper on operational IP networks, the researchers did not treat a traffic matrix as a table waiting inside a router. They built it as a disciplined reconstruction from several data sets, each carrying a different part of the answer.

The distinction is subtle and operationally decisive. Link load is an outcome: traffic demands passed through a particular topology under a particular routing configuration. A demand model must survive a hypothetical change to that configuration. Without it, traffic engineering risks using yesterday’s route to explain tomorrow’s load.

A demand has an entrance, a volume and possible exits

An end-to-end matrix for every source and destination address would have been vast. It would also have demanded visibility that no single provider possessed. Much Internet traffic crossed several administrative domains; an ISP could see the part inside its own backbone, not the whole journey.

The team chose a more useful abstraction. A demand was a volume of traffic entering at an ingress link and destined for a set of egress links able to reach the same destination prefix. The set mattered. A BGP policy or a withdrawal could alter which exits remained possible, while an internal routing change could select a different member of the set. The demand did not have to be redefined each time an OSPF weight changed.

This point-to-multipoint model kept two objects separate. The demand described the offered traffic. Routing and topology described how that traffic became load on internal links. NetScope, the team’s related traffic-engineering system, could then locate a congested link, identify the demands crossing it and simulate a new configuration before an operator touched the live network.

The model also set a boundary on precision. Traffic engineering operated over tens of minutes, hours or days, not at the timing of each packet. Flow records could be divided into time bins and aggregated by ingress and possible egress set. The aim was not to reconstruct every conversation. It was to retain enough structure to reason about the consequences of network decisions.

Four records had to meet at the same moment

The ideal collection point was every ingress link. A flow record there supplied the input interface, destination address, start and finish time, and byte count. A forwarding table linked the destination prefix to possible egress links. Router configuration files supplied link names, roles, capacities, packet filters and OSPF parameters. Topology and routing logic connected the entrance to the exits.

In the AT&T backbone, full flow measurement on every customer access link was not practical. Some routers lacked the feature; enabling it elsewhere could consume scarce resources. The researchers adapted by collecting primarily at peering links, where capable routers handled a large share of inter-provider traffic.

That economy created known blind spots. Access-to-access traffic inside the provider could pass without touching a peering link. Outbound traffic was observed when it left, so its original access link might be unknown. Multihop transit traffic could appear once at entry and again at exit, creating a duplicate unless the second record was excluded.

The outbound case shows why the matrix was inferred rather than observed. A source address could be associated with several customer access links. The routing model tested each candidate: would traffic entering there, under the topology and configuration captured for that time, have reached the peering interface where the flow was seen? Impossible candidates were removed. Sometimes one ingress remained. Sometimes several remained and the volume was shared among them. Sometimes no route was consistent, and the record became a miss.

The output therefore carried epistemic states, not just numbers. “Observed at this egress,” “consistent with one ingress,” “consistent with several” and “inconsistent with the snapshot” were different claims. Converting all four to a single confident cell would have erased the very information an operator needed to judge the model.

A measurement stream can lose ninety per cent and still look busy

The most instructive failure occurred between the routers and the collection server. NetFlow records were exported in UDP packets. During heavy periods, limited bandwidth on the collector link caused losses as high as 90 per cent. A graph built from received records alone would still rise and fall. It would simply report a dangerously incomplete scale.

Sequence numbers allowed the team to examine the missing exports. The observed pattern was consistent with approximately independent loss, so they estimated a loss probability for each ten-minute period and applied a correction factor to flows received during that interval. This did not recreate the missing records. It scaled an incomplete sample under a stated statistical assumption.

The check came from a different instrument. SNMP counters reported bytes on each interface at five-minute intervals. After loss correction, NetFlow-derived utilization tracked the corresponding SNMP curves relatively closely. SNMP could not supply the ingress-to-egress matrix, but it could challenge the reconstructed total at a link. The validation was useful precisely because it did not reuse the same failed export stream.

The four data families also had to agree about identity and time. A NetFlow record named interfaces by integer SNMP indexes; configuration and forwarding files used names and addresses. Router clocks on interface cards were not always aligned with the route processor. Configuration files, forwarding tables and counters were captured at different hours. The researchers had to join identities, correct clocks and keep snapshots close enough to the traffic they interpreted.

One anomalous day exposed the cost of getting that join wrong. An access link had been upgraded after the configuration snapshot, while a later forwarding table already referred to the replacement. Customer prefixes no longer matched the old link identity, and unexplained misses rose. Reconciling the change brought the miss rate back toward the other days. The lesson was not that manual fixes make inconvenient evidence disappear. It was that a mismatch can reveal a real change in the network’s state.

The residuals belonged in the result

Across four November 1999 experiments, the aggregation generally mapped more than 98 per cent of bytes seen at peering links to a demand form. For outbound traffic it found at least one candidate ingress for more than 99.3 per cent of bytes. Those high figures did not mean unique attribution. Roughly 35–45 per cent of outbound bytes initially had several candidate ingress links.

Routing eliminated some alternatives. About one-quarter to one-third of the ambiguous volume could be resolved to one ingress in the reported runs. Several candidates often remained when a customer had redundant access links in the same city, where the corresponding backbone paths were similar. After disambiguation, roughly 2.5–4 per cent of outbound bytes still could not be placed into one or more point-to-multipoint demands.

Those percentages belong to four days on one late-1990s backbone. Their lasting value is not as a benchmark. It lies in the method’s refusal to confuse coverage with certainty. A model could cover most bytes while remaining ambiguous about the source of a substantial share. A consistency check could pass almost all inbound traffic while still exposing stale reachability at exactly the moments routing changed.

The traffic analysis found a small population of heavy demands carrying much of the volume, alongside strong time-of-day variation and some stability in high-ranking demands from one day to the next. That pattern suggested selective measurement might capture much of the operational value. It also warned that concentrating major demands on common links could turn a routing mistake or a change in one source’s behavior into a large load shift.

The matrix was an argument with the network

Feldmann’s official research record links traffic analysis, modeling and routing because the three could not be separated in this work. Measurements without routing context described effects but could not support a new routing decision. A routing model without measured demand produced elegant paths for imagined traffic. Configuration without time alignment could confidently explain a network that no longer existed.

The traffic matrix joined those partial views. It remained a model, dated by its snapshots and qualified by loss, ambiguity and residuals. Yet that qualified model gave operators something a red link never could: a way to ask where the load came from, to test where it might go, and to see which parts of the answer were observed rather than inferred.

That is a demanding standard for modern observability. Keep the raw counter, but also keep the configuration and route state that made it meaningful. Record the correction rather than replacing the sample silently. Retain the unresolved candidates. Challenge the reconstruction with an independent aggregate. A dashboard can show a link. Decision-grade evidence must explain the path from demand to that link—and the conditions under which the explanation stops being true.

Sources