Summary

  • Rapport’s 4 September 2026 change split a missing-manifest test into two cases: a changed CA instance and the same CA instance with usable cached data.
  • The same-instance case verifies that six expected VRPs survive, including two retained from cache after ca2.mft is removed.
  • Its source says the validator must warn, but the only warning assertion is commented out; the test does not execute that check at the pinned commit.
  • RFC 9286 treats cached continuity and a reason-bearing operator warning as separate requirements. A complete acceptance receipt must prove both.

The most interesting line in LACNIC’s Rapport test is the line that does not run.

The scenario is deliberately ordinary for a failure test. A relying party first builds a known-good cache. Rapport then creates another repository state, removes one certification authority’s manifest and runs the relying party again. The test expects the old validated payload from that CA to remain available. Immediately before the output assertion, a comment states what else should happen: the validator must warn about the missing manifest. The check_logfile command underneath begins with a shell comment character. It is inert.

What follows is active. Rapport checks for six validated route-origin payloads, or VRPs. Two belong to the first CA, two to the CA whose manifest is now unavailable, and two to a newly introduced sibling. In other words, the test asks a useful and exact continuity question: did the validator preserve the cached output associated with the affected CA while continuing to process the rest of the repository?

That is not a weak test. It captures one of RPKI’s least intuitive safety properties. A failed fetch need not mean that previously validated routing data should vanish immediately. Abrupt removal can be worse than cautious retention. RFC 9286 therefore tells a relying party to terminate processing for the affected CA instance, warn about the reason and continue using cached objects associated with that instance until they become stale or can be replaced after a successful fetch.

The problem is not that Rapport preserves the cache. The problem is that its pinned public test proves only half of the bargain.

One failure, two observable duties

An RPKI manifest is a signed inventory. It lists the certificates, current revocation list and signed objects associated with a CA’s publication point. It lets a relying party detect missing, changed or replayed material that signatures alone cannot expose. A manifest fetch can fail because the object cannot be retrieved, because it is invalid or stale, or because a file it names cannot be acquired or does not match its recorded hash.

RFC 9286 separates the response into two duties. First, the software must preserve a controlled view of routing state. Section 6.6 says processing for that CA instance ends and recommends continuing to use its previously cached objects, subject to freshness and later successful replacement. Second, the software must surface the degradation. It says the relying party must issue a warning that identifies the reason for termination and recommends that a human operator be notified.

Those duties serve different people. Cache retention protects the router-facing result from an avoidable cliff. The warning protects the operator-facing account of how that result was obtained. A table of VRPs may look perfectly normal in both a fresh successful cycle and a failed cycle supported by prior cache. Without a separate diagnostic, the result is continuous but its evidence state has changed.

This is why “the expected routes are still present” cannot stand in for “the failure was made visible.” The payload assertion answers what the relying party supplied. The warning assertion answers whether someone can distinguish a current repository view from a deliberate fallback. Both can pass. Either can fail without forcing the other to fail.

The frozen evidence does not show that FORT or another relying party omitted a warning. It does not show that the test fails. It shows something narrower and fully inspectable: in the same-instance fallback script at Rapport commit c682fd2, the source comment calls for a warning, while the command that would match a specific FORT log message is commented out. A shell will not execute it.

Rapport’s split makes the identity boundary visible

The 4 September change is valuable because it replaced one broad scenario with two more precise ones.

In the first, a CA certificate is replaced and its SIA manifest URI points somewhere that does not contain the manifest. Rapport’s suite notes describe this as a new CA instance with no prior cache applicable to it. The active script checks for a FORT diagnostic and expects the affected CA’s earlier VRPs to be absent while sibling and newly added CA output remains.

In the second, the CA certificate stays the same. Its SIA and publication point are unchanged. Rapport removes that CA’s manifest after a successful first cycle and expects the cached VRPs to survive the second cycle. This script contains the disabled warning check.

The distinction is not a cosmetic fixture change. RFC 9286 says manifest processing occurs separately for each CA instance, guided by the manifest URI in the CA certificate’s Subject Information Access extension. Multiple CA instances can even share one publication point during rollover; their manifests still describe separate collections. Cache fallback belongs to the affected CA instance, not to a directory name in the abstract.

That identity boundary is the reason the two tests should not be collapsed. A validator must not borrow old objects from a different CA instance simply because their files occupy a familiar place. At the same time, it should not discard a still-usable cache for the same instance merely because a later fetch failed. Rapport’s revised suite makes this operating difference unusually concrete.

It also makes the warning gap more consequential. A fallback event is not just “some download failed.” The diagnostic needs to bind the failure to the CA instance whose processing stopped, the cached state that remained authoritative for the moment, and the conditions under which that state will expire or be replaced. If the payload is preserved but the reason is not testably visible, continuity can masquerade as freshness.

A literal log line is the wrong long-term contract

Simply removing the comment marker may not be enough. The disabled assertion searches for one exact FORT sentence containing a local manifest URI. That can be useful while developing one adapter, but it is a brittle interoperability contract.

Rapport’s README says the project can test fort2, Routinator, rpki-client and rpki-prover, while also warning that the tests are presently built around FORT. Each implementation may expose the same semantic event differently. One may write a text log; another may emit structured JSON, increment a metric, return a status object or use different terminology for the same CA-instance failure. Exact wording can change between versions without changing the safety property. Conversely, a familiar phrase can appear while omitting the identity or fallback facts an operator actually needs.

The portable assertion should therefore be semantic. For this scenario, it should establish at least seven things: a manifest-acquisition failure occurred; the reason class is identifiable; the affected CA instance is identifiable; the relying party selected cache fallback; the age or freshness boundary of that cache is available; the event was emitted during the expected test step; and the configured operator-notification path received or deliberately suppressed it.

Implementation adapters can prove those facts through different surfaces. The FORT adapter may parse a log pattern. Another adapter may query metrics or structured output. Rapport does not need to impose one sentence on every relying party. It needs to make the expected meaning stable while keeping evidence of how each adapter observed it.

The same approach would improve the cache assertion. Six displayed VRPs prove the expected final set, but a stronger receipt would also bind them to the prior successful cycle. It would record the CA-instance fingerprint, previous fetch time, cached-object digest, failed-fetch reason, retained VRP digest, sibling processing result and next retry. That turns “the numbers happen to match” into “these records survived for the reason RFC 9286 permits.”

The acceptance receipt needs two columns

A compact test receipt can keep payload and observability distinct.

The first column is the cache decision. It names the repository and Rapport commit, the scenario, the relying-party product and binary digest, the CA certificate and SIA fingerprints, the last successful fetch, the manifest that became unavailable, the cached-object set, its freshness limit, the resulting VRP set, sibling-CA isolation and the next retry state.

The second column is the operator signal. It records the semantic event identifier, reason class, affected CA-instance reference, fallback decision, cache age, severity, emission time, adapter and evidence path. It also records whether a human notification was expected, delivered, rate-limited or intentionally disabled by the test configuration.

The receipt should refuse to merge these columns into one green result. Payload continuity can pass while warning visibility fails. A warning can pass while the wrong cached set survives. A supported implementation may not expose a machine-readable human-notification state; that should be an explicit untested or unsupported disposition, not an assumed success.

This is not paperwork added after the test. It is the test’s public meaning. A vendor, operator or standards participant looking at a result must be able to tell which safety property was observed and which was not. A count without that boundary can be carried into release notes or procurement discussions long after the console context has disappeared.

Why a quiet fallback is an operating risk

The immediate technical advantage of cache fallback is stability. Routes derived from previously validated objects do not disappear because of a transient repository or transport problem. RFC 9286 explicitly explains that this can prevent the relying party from misinterpreting data and treating invalid routes as valid, or valid routes as invalid.

The operational cost is ambiguity. The downstream router cannot tell from the retained VRPs alone that the upstream evidence refresh failed. The NOC may see no route-count movement. A dashboard that measures only payload availability can remain green. If the warning surface is absent, untested or disconnected from human attention, the first visible symptom may arrive when cached data finally becomes stale, when another incident overlaps, or when an operator tries to explain why two validators disagree.

The purpose of the warning is not to create alarm theatre. It is to assign a degraded but intentional state. “Serving validated cache after manifest-fetch failure” is neither full health nor immediate failure. It needs an owner, an age and a next action. Too little signalling hides evidence decay. Too much undifferentiated signalling teaches operators to ignore it.

That balance is precisely why the test should assert semantics rather than noise. One event per affected CA instance and failure episode can carry the reason, cache state and retry transition. Repeated failed polls can update duration without producing an endless stream of indistinguishable alerts. Recovery should close the same event when a successful manifest fetch replaces the cache. The resulting history tells an operator not only that a warning once appeared, but whether the condition persisted and ended.

What the source does and does not establish

Rapport remains early-stage software. LACNIC’s own introduction says Barry and Rapport had a long way to go. The repository contains no public release or tag in the frozen capture, and active commits show the suite still changing. That context argues for careful evidence, not dismissal. Early test software is exactly where the boundary between an expected result and a claimed guarantee should be explicit.

The commented line may reflect unstable wording, an adapter limitation, unfinished work or a temporarily disabled assertion. The public sources do not say. It would be wrong to infer a validator defect or maintainer intent from a comment character. It would be equally wrong to report the warning behaviour as tested when the check does not run.

The fairest conclusion is also the most useful. Rapport’s new scenario already protects the hard half of continuity: it checks which routes remain and isolates the affected CA. Its source already names the other half. Completing the test means turning that intention into an implementation-neutral assertion and preserving both outcomes in one receipt.

Cached routes answer whether the network view stayed usable. The warning answers whether anyone knows why.

Sources