Summary

  • ARIN awarded Virginia Tech US$17,350 for an RPKI Operational Health Dashboard. The recipient-provided summary cites a 14-year warehouse with about 3.9 billion object observations and 139 GB of Parquet data.
  • The same summary reports revocation compliance from 9% at AFRINIC to more than 99% at ARIN since 2023, plus an ARIN re-signing-cadence shift from 14 to 80 days. It says the analysis is under review at IMC 2026.
  • The public grant page does not define the compliance numerator, denominator, observation window or exclusions. The existing beta API defines some neighbouring metrics, but does not publish a reproducible crosswalk for those headline figures.
  • The dashboard should attach a versioned measurement receipt to every RIR comparison, so a change in data, code or population cannot masquerade as a change in institutional behaviour.

Two scale figures, two judgement figures

ARIN’s detailed recipient page labels its descriptions as project summaries provided by grant recipients. That attribution matters. Virginia Tech says it has built a 14-year longitudinal warehouse spanning all five Regional Internet Registries, containing about 3.9 billion observations of RPKI repository objects in 139 GB of Apache Parquet.

Those are scale claims. They describe how much material was observed and stored. The next figures do something different. The summary says CRL validity windows vary sevenfold, re-signing cadence varies sixteenfold, and “revocation compliance” ranges from 9% at AFRINIC to more than 99% at ARIN since 2023. It also says ARIN’s own 2023 changes are visible as a move from 14 to 80 days in re-signing cadence, a reduction in CA-certificate lifetime from ten years to 825 days, and a jump in revocation compliance from below 5% to more than 99%.

A warehouse can be counted by bytes and observations. A compliance percentage requires a rule. It needs a population that could comply, an event or object that counts in the numerator, a time window, treatment of missing observations, and a definition of failure. Without those elements, 99% looks more finished than the public evidence behind it.

This is not a reason to dismiss the result. It is a reason to preserve its identity. The summary says the underlying analysis is under review at IMC 2026. “Under review” is neither rejection nor acceptance. The grant is also not peer review. ARIN’s 22 September announcement records a funding decision and project title; it does not claim that ARIN reproduced the analysis.

The beta surface shows why names are not enough

The linked RPKI Warehouse describes itself as a beta longitudinal observatory using daily snapshots from all five RIRs. It offers ROA lookup, lifecycle, coverage and organization views and warns that data and features may change. This may be an existing research surface rather than the completed grant deliverable. It should not be judged as if the funded weekly dashboard were already finished.

It is still useful evidence about metric identity. The public OpenAPI schema defines “signing cadence” as the interval between consecutive CRL this_update timestamps for each CA, identified by its Authority Key Identifier. On 25 September, the unfiltered ARIN cadence response reported a median interval of 24 hours.

The recipient summary, meanwhile, describes an ARIN move from 14 to 80 days. Those figures may both be correct. One may concern CA certificates, repository products, historical eras, selected CAs or a different aggregation. The 24-hour response may describe current CRL issuance while the 80-day figure describes another re-signing object. An apparent contradiction is therefore not evidence of error. It is evidence that the metric name alone does not preserve the question.

The API is more explicit about CRL staleness: it considers currently live CRLs and calculates staleness as the current time minus next_update. That is a clear operational definition. It is not the same as revocation compliance. The inspected schema did not expose a public endpoint or formula named revocation compliance, so a reader cannot yet reconstruct the 9% and 99%+ comparison from the grant page and beta API alone.

A CRL is a control object, not a percentage

RFC 6487 profiles RPKI resource certificates and Certificate Revocation Lists. Each CA issues a CRL covering its certificates; the current CRL helps a relying party decide whether a certificate in a validation path has been revoked. The RFC specifies required fields and validation behaviour. It does not define one cross-RIR compliance score.

RFC 9286 adds another time boundary. A manifest lists the signed objects at a CA publication point, including its most recent CRL, and carries thisUpdate and nextUpdate. A relying party checks whether the manifest is current, whether listed files can be obtained and whether their hashes match. After a failed fetch, it may continue with cached material until that material becomes stale or a later fetch succeeds.

That means an observation can occupy several states. A CRL may be present in a repository snapshot, listed on a manifest, current by its own time fields, retrievable by one collector, missing for another relying party, or associated with certificates whose lifecycle changed between snapshots. “Repository-present” and “RP-valid” are not interchangeable populations.

A comparative score must say which state it counts. It must also say whether the unit is a CRL, a CA, a publication point, a snapshot, a revocation event or a certificate that should have been revoked. If a CA produces hundreds of observations, an observation-weighted percentage can give it far more influence than a CA-weighted percentage. If hosted and delegated CAs are assigned to an RIR in different ways, the apparent regional result can change without any operator changing behaviour.

The denominator is a governance decision

Consider three plausible meanings of revocation compliance. The first could ask whether a superseded certificate appears on the next CRL. The second could ask whether every revoked, non-expired certificate appears on the current CRL. The third could ask whether a new CRL was published within a required interval after an event. Each could produce a percentage. They do not measure the same control.

The denominator could be all revocation opportunities, all CAs, all snapshots, or all eligible certificates. Missing data could be classified as non-compliance, excluded as unobservable, or carried forward from a cached successful fetch. A long outage could therefore lower a score, remove observations from it, or leave it unchanged depending on one methodological choice.

These choices are not merely statistical housekeeping. A dashboard can influence operator trust, grant priorities, audit attention and institutional reputation. If one RIR is shown at 9% and another above 99%, readers will reasonably infer an enormous difference in discipline. The publisher owes them enough information to tell whether the difference comes from conduct, architecture, collector coverage or the scoring rule.

ARIN’s grant can create precisely that public good. A shared dashboard is more valuable than a static paper when it lets operators inspect fresh conditions, compare history and test improvements. The condition is that refreshability must not erase provenance. A weekly number without its version history can silently change meaning while preserving the same label.

Give every score a measurement receipt

A compact receipt could begin with a stable metric ID and semantic version. It would state the plain-language question and the exact object or event being tested. It would name the applicable RFC profile, the collector, repository snapshot and observation window. Then it would publish numerator, denominator, unit of analysis, inclusion and exclusion rules, deduplication, RIR-attribution rule and treatment of failed fetches, expired objects, missing snapshots and cached data.

The receipt should distinguish hosted from delegated CAs, repository presence from relying-party validity, and current measurements from historical backfills. It should identify the code and data release, expose a reproducible query where possible, record known limitations, and keep a correction log. A digest of the exact result set would let later researchers prove which number they cited.

Uncertainty can be visible without making the dashboard unreadable. A headline might show “99%+ under metric RC-2.1, 1 January–31 December 2025,” with the population count and a link to the recipe. If a collector gap makes part of the week unobservable, the interface can mark coverage rather than forcing missing evidence into either success or failure.

The goal is not to burden an early dashboard with ceremonial paperwork. It is to keep the most valuable property of a 14-year dataset: the ability to distinguish a real operational change from a changed lens. The recipient summary says ARIN’s 2023 shifts are directly visible in the data. A versioned receipt would let the public see exactly where that visibility begins and what it excludes.

Sources

  1. ARIN — Congratulations to the 2026 ARIN Community Grant Recipients
  2. ARIN — Community Grant Program Recipients
  3. RPKI Warehouse beta
  4. RPKI Warehouse OpenAPI schema
  5. RPKI Warehouse — ARIN signing cadence
  6. RPKI Warehouse — ARIN CRL staleness
  7. RPKI Warehouse — data health
  8. RFC 6487 — A Profile for X.509 PKIX Resource Certificates
  9. RFC 9286 — Manifests for the RPKI