Summary

  • Craig Johnson’s NANOG 98 lab case removed le 32 from a BGP export prefix list. The session stayed established, but neither advertised /25 matched the revised rule.
  • His emulator observed the network software fail inside a constructed lab; a behavioral twin computed which paths would disappear from a collected network model. Those are different pieces of evidence.

The BGP session was still up. That was the trap. In Craig Johnson’s example, a tidy edit changed permit 10.20.30.0/24 le 32 to permit 10.20.30.0/24. The service sent two /25 routes, and neither matched the narrower rule. The lab’s HTTPS probe timed out. A dashboard watching only session state could have passed the change with a green light and a broken service.

Johnson presented the case at NANOG 98 in Miami on 19 October. He had built Skyforge, an internal platform that turns a declared network into disposable labs. A YAML topology goes through netlab, KNE and meshnet, then boots real network operating system instances. That lets him ask whether a command is accepted, a protocol converges, and an actual virtual device withdraws a route. In this example the lab saw the /25s disappear while BGP stayed established.

He then asks a second question: what does the change do in the network that the operator has collected? A behavioral digital twin computes paths from configuration and state pulled from that network. For his example, the modeled path disappears and eight of 56 source-destination pairs change. That is wider scope than the eleven-node lab. The model can identify policy consequences across the captured network; it cannot claim to have run the same NOS transition or proved the application’s response time.

The distinction cuts both ways. A perfectly behaving emulator can be a perfect answer about the wrong topology. It does not automatically include external peers, current failures, return paths or learned routes that were never built into the lab. A model may cover those collected relationships while missing an implementation quirk or a transient failure. Johnson’s two instruments agree on this prefix-list case. Their agreement is evidence for this change, not a universal certificate for either instrument.

His deck supplies useful scale limits too. One run brought 100 cEOS nodes to Ready in 374 seconds; a 250-node attempt stopped at 220 when memory ran out. After a fix, 500 of 500 nodes were Ready in 925 seconds. These are his lab measurements. They explain why the broad network question belongs to a collected model rather than a habit of booting an ever-larger copy of everything.

The proposed change sequence writes the edit in git, applies it to lab devices with undo armed, takes a fresh snapshot, and keeps the edit only if the checks pass. The slides state this workflow ran on lab devices. The lesson is sharper than the word “digital twin”: demand the observation that proves the device behavior, and the model result that proves the affected paths were in scope.

Sources: NANOG 98 session · Johnson’s original slides.