Summary

  • Version 04 of the IETF SAV benchmarking draft requires reports to identify the packets treated as legitimate or spoofed and explain the basis for those labels.
  • Its false-positive and false-negative rates use different packet populations, so a disputed legitimacy label changes the meaning of the reported result.
  • The document describes repeatable laboratory measurement, not a pass/fail conformance regime, a new SAV protocol, or an operational deployment recommendation.

An accuracy table can look objective while concealing its most consequential editorial choice. If a test sends 100 packets called legitimate and a device blocks two, the reported false-positive rate is two per cent. But the arithmetic settles nothing if the two packets came from a hidden prefix, a limited-propagation prefix, a direct-server-return path, or another source that the test harness mistakenly treated as spoofed. The label decides whether those drops are security successes or availability failures.

That is the important change made explicit in draft-ietf-bmwg-savnet-sav-benchmarking-04, posted on 14 September 2026. The draft says an accuracy report should identify which packets in every test are legitimate and which are spoofed, and state why. It also sharpens the document’s scope: the benchmark records improper blocks and improper permits; it does not establish a pass/fail threshold for conformance.

The distinction follows the two measurements. The false-positive rate counts legitimate packets incorrectly blocked by the device under test, divided by the legitimate packets sent. The false-negative rate counts spoofed packets incorrectly permitted, divided by the spoofed packets sent. These are not interchangeable percentages. They describe different harms, use different denominators and depend on different classifications made before the counter is read.

A label assembled from context

Source address validation asks whether traffic arriving on an interface has a source address that should be accepted there. The word “should” carries more context than an address-to-route lookup can always provide. The draft’s test cases account for interface location, the business relationship represented by that interface, the routing and SAV information available to the device, local configuration and the scenario being exercised.

That matters at network edges where legitimate reachability is not fully visible in ordinary BGP announcements. The draft discusses hidden prefixes and limited prefix propagation, both of which can carry authorised traffic without appearing to every observer in the same way as a globally propagated route. A direct-server-return design can also produce a return packet whose source does not fit a simplistic symmetric-path assumption. Customer, provider, peer and route-server-facing interfaces create different expectations again.

The benchmark therefore does more than instruct a tester to send “good” and “bad” traffic. It asks the report to preserve the conditions under which those judgments were made: the deployment and topology, the tested interface and its relationship, the device version and configuration, the sources and update timing of routing or SAV information, traffic characteristics, timestamps, repetitions and statistical treatment. A reader needs those facts to determine whether a later comparison is genuinely like-for-like.

The test is deliberately black-box. The device under test may be a hardware router, a software router, a virtual machine or a container. The method observes what is forwarded or blocked without requiring a particular implementation. That neutrality is useful, but it increases the importance of the external truth used to label the packets. A black-box counter cannot explain a bad label supplied by the laboratory.

The draft also separates SAV outcomes from ordinary forwarding loss. A packet that disappears because of congestion, an unrelated filter or a broken forwarding path is not automatically evidence of a source-validation false positive. Conversely, the intentional blocking of spoofed traffic should not be counted as generic packet loss. Mixed tests must preserve a distinct account of legitimate forwarding if their results are to remain interpretable.

What the draft does not certify

The status of the work is as important as its detail. Version 04 is an active BMWG working-group Internet-Draft intended for an Informational publication. It is not an RFC, has not been approved by the IETF, and reports no product results. The BMWG milestone currently points to a January 2027 submission to the IESG. That schedule is a work-plan marker, not an assurance of publication or adoption.

Nor does the draft define a universal acceptable error rate. Two laboratories could follow the method and produce different numbers because the devices, information inputs, topologies and traffic populations differ. The value of the method lies in making those conditions visible enough for the difference to be investigated. Calling one result “compliant” would require a policy threshold that this document does not supply.

That restraint is consistent with the relationship between benchmarking and protocol work. SAVNET documents the intra-domain and inter-domain problems that source address validation mechanisms must address. BMWG defines how performance or accuracy can be measured in a controlled setting. Measurement can expose operational trade-offs, but it neither invents a new SAV mechanism nor decides which mechanism an operator should deploy.

Sources