Summary

  • Robert Kisteleki’s 2023 RIPE NCC proposal sought to preserve as much RIPE Atlas and RIS history as financially sustainable while allowing older or rarely requested results to move to slower access tiers.
  • The design kept collected results available, proposed different protocols and time-to-access targets by age and service, encouraged authenticated requests, and promised owners at least one month of access to non-public Atlas results.

The archive grows up

Every Internet measurement has a timestamp. A timestamp, however, does not pay a storage invoice. Robert Kisteleki had helped build RIPE Atlas and RIPEstat and later led RIPE NCC research and development. In November 2023, with contributor Paul de Weerd, he brought a less romantic question to RIPE’s measurements community: how long should an archive behave as though yesterday’s packet and a result from 2010 deserved the same fast doorway?

The proposal arrived with scale attached. RIPE RIS had collected routing data since 1999 and RIPE Atlas active measurements since 2010. At the end of 2023, Atlas was producing about 15,000 results each second, roughly 1.3 billion daily. More than 90% of its measurements were one-off runs; ongoing measurements, though fewer, delivered most of the results. An operator checking an outage wants the newest answer. A researcher revisiting a decade of routes may need the oldest one. Both enter through an interface somebody operates.

RIS held about 50 TB of compressed dump and update files at that time, with 80% gathered in the preceding five years. RIPEstat-oriented forms of the data occupied about 800 TB. Those numbers are different kinds of storage, yet together they show the cost of making history convenient. Kisteleki’s answer was to keep as much of it as finances allow, while setting different access protocols and time-to-access targets for fresh and old data. Older, rarely used results might move out of the original API or take longer to retrieve. Atlas and RIS could have separate cutoffs.

Authenticated users might receive different transfer treatment; anonymous traffic could share a pooled bandwidth limit. Owners of non-public Atlas measurements would be able to retrieve their results for at least a month.

The remarkable move was putting latency inside the public-data bargain. A file can exist and still be hard to use during an incident. Conversely, a historical archive with slower retrieval can remain useful if its access path and timing are explicit. Kisteleki offered neither a universal cutoff nor a finished service-level number. His RIPE 87 slides placed value and operating cost on the same agenda. In a RIPE-hosted May 2024 discussion conclusion, the team said feedback had been limited and it was implementing storage and access features along the principles.

That makes Kisteleki an unusual custodian of the Internet’s memory. He did not offer a bonfire for old measurements. He put a price, a clock and a visible access rule next to the cupboard. Anyone who has waited for a historic trace during a live outage knows why all three matter.

RIPE originals