Summary

  • The Energy Sciences Network (ESnet) is a scientific network facility of the U.S. Department of Energy Office of Science, operated by Lawrence Berkeley National Laboratory. It is not a commercial telecommunications provider or an independent corporation; it is federally funded scientific infrastructure.
  • ESnet6 combines approximately 15,000 miles of dedicated fibre base, optical transmission, packet routing, private networks, interconnections with cloud and research and education networks, measurement, and programmable services. The 57 Tbps reported in 2024 refers to aggregate capacity; the 1.77 exabytes is annual traffic, and does not imply that all users have 57 Tbps paths available.
  • ESnet's most important contribution has been treating network performance as an end-to-end problem. Science DMZ, perfSONAR, iperf3, OSCARS, SENSE, High-Touch, and EJFAT improve local systems, network reservations, telemetry, and live data processing, linking backbone capacity to tangible scientific outcomes.
  • The next phase is about more than raw speed. American Science Cloud, Integrated Research Infrastructure, plans for ESnet7, real-time instrument streaming, field wireless, and quantum network control aim to embed the network directly into scientific workflows—though maturity varies, and plans and demonstrations cannot be treated as widely available production services.

Not a retail ISP, but a network for science

To understand ESnet, the easiest way is to follow the path a piece of scientific data takes. Detectors, telescopes, microscopes, and simulations generate information at research facilities; collection systems receive it; storage and transfer servers prepare it for movement. Local networks carry data to a managed boundary; beyond that, ESnet transports it within the United States, to research facilities across the Atlantic, to DOE supercomputers, to research and education networks, and to commercial clouds. What returns may be analysed data, alerts, models, or information to steer the next operations of an experiment.

No single organisation controls this entire path. Instrument teams, laboratory LANs, regional research networks, cloud providers, international partners, and supercomputing facilities are each likely separate operators. ESnet provides the wide-area network layer dedicated to science missions and works with the other operators to make the complete path function. Slow storage, congested campus links, or inappropriate firewalls can waste a fast national backbone, and a failure on ESnet's side can halt a well-designed local facility.

Its institutional position is also clear. ESnet is a user facility of the DOE Office of Science, primarily funded and overseen by Advanced Scientific Computing Research (ASCR) and operated by the Scientific Networking Division of Berkeley Lab. Berkeley Lab itself is managed by the University of California under contract with the DOE. ESnet has no shareholders, no publicly traded stock, no independent enterprise value, and no standalone financial statements as a commercial entity. Consequently, it should be described as federal science infrastructure, not as a company selling network services.

Researchers do not ordinarily contract directly with ESnet. Instead, in the course of using scientific instruments, supercomputers, laboratory services, university networks, or collaborative research networks, their traffic transits ESnet. ESnet distinguishes between institutional site users and endpoint users; it does not formally register or track individual end users as subscribers. The “over 30,000 organisational users” cited in the 2024 annual report is a snapshot of institutional scale, not a count of individual subscribers.

Given this operating model, comparisons with conventional telecoms companies alone miss ESnet's role. While ESnet operates optical transmission, routing, and interconnections, it also engages in requirements reviews, open-source development, experimental services, and site performance improvement. What it delivers is not retail circuits. It delivers the ability for distant scientific facilities to function as one shared system. (ESnet Governance;Network Services)

Origins in scarce mainframes and acoustic modems

ESnet's official history stretches back to the mid‑1970s, when both computers and long‑distance communications were scarce. At Lawrence Livermore National Laboratory's Controlled Thermonuclear Research Computer Center, a borrowed Control Data Corporation 6600 was connected via four acoustic modems. The equipment and speeds belong to another era, but the operational problem remains familiar: a specialist scientific community needed remote access to expensive computational resources that no single institution could replicate.

From the late 1970s into the early 1980s, different research disciplines within the DOE built their own networks. High‑energy physics and magnetic fusion had different facilities, different collaborators, and different data flows; HEPnet and MFEnet reflected those boundaries. When commercial services could not provide the reach, performance, and operational support required, dedicated networks made sense. However, each programme procured circuits independently and sustained its own technology and expertise, resulting in duplicated investments and operations, and fragmented long‑term renewal and cross‑discipline coordination.

The formal launch of ESnet in 1986 consolidated these functions into a broader science network. The change was as much organisational as technical. A shared facility could forecast demand across multiple programmes, operate national circuits, coordinate international connections, and maintain specialist personnel. Capacity shifted from being a problem for individual experiments to being an infrastructure planning concern for the entire DOE.

That early model already carried features that continue today. Users were defined by science mission rather than by a retail market; central mainframes and scientific instruments justified the shared public investment. Network design followed long‑term research plans, and operation required staff who understood both communications technology and the scientific impact of failures. Demand was not uniform; a single experiment could produce traffic far exceeding normal levels.

Berkeley Lab also had its own networking research lineage. In 1974, it connected a CDC 6600 to the ARPANET; later, researchers contributed to foundational work on TCP congestion control. The achievements of Van Jacobson and Mike Karels belong to Berkeley Lab's wider research environment and should not be treated as ESnet's alone. Nevertheless, a culture of treating protocol behaviour, measurement, and performance as researchable engineering challenges sat close to ESnet's operational ethos.

ESnet operations moved to Berkeley Lab in 1996. This placed the current operational home in the same environment as the National Energy Research Scientific Computing Center (NERSC), network researchers, software engineers, and other DOE programmes. The present arrangement—in which a production network and an applied research organisation share staff, research facilities, and technical challenges—became established here. (ESnet History)

Why a network became a scientific user facility

Scientific user facilities exist to share capabilities too expensive, too specialised, or too interdependent for any single research group to build alone. Like particle accelerators, light sources, and supercomputers, ESnet applies this concept to communications. It pools long‑haul fibre rights, optical equipment, routers, international capacity, around‑the‑clock operations, cybersecurity, performance measurement, and technical support into one facility used by multiple science programmes.

The shape of science demand explains the need for public funding. Commercial telecoms providers can sell high‑capacity circuits, but they may not design a national network around detectors that will begin operating years later, supernovae that may not occur during a contract period, or scientific workflows whose traffic is modest but whose public value is high. ESnet can invest before measurable demand materialises, because its mission is to deliver scientific capability, not short‑term network revenue.

The user‑facility model also changes how demand is understood. ESnet conducts formal requirements reviews with DOE science programmes. Researchers and facility teams describe instruments, data volumes, storage locations, computation destinations, time constraints, collaboration structures, and local bottlenecks looking five to ten years ahead. The resulting intelligence informs decisions on site connection capacity, path diversity, trans‑Atlantic capacity, orchestration research, and service design.

This work draws less attention than a new optical link, but it may be more important. Submarine spectrum contracts, router procurement, or a second physical entrance to a site can take years from commitment to deployment. If an experiment is already producing data before such preparations are made, a predictable infrastructure need becomes an emergency. Requirements reviews convert science plans into engineering lead‑times.

At the same time, requirements reviews make uncertainty explicit. Instrument schedules slip, data‑reduction software improves, cloud usage expands, and facilities may redesign workflows. ESnet does not treat a single case study as a locked traffic order. It builds multiple scenarios and determines where flexibility, headroom, and phased delivery are needed.

In this model, individual researchers are not charged per gigabyte. Core operations are federally funded; some site connection costs may be borne by programme sponsorships or under the Site User Cost Policy. But incentive differences remain: sponsoring programmes may want low connection charges; ESnet seeks capacity and resilience that benefits multiple programmes; sites may defer local upgrades needed to exploit the national network fully. The governance role is not to shift costs to a different budget line, but to tie such decisions to scientific outcomes. (ESnet Site Coordinators Committee and cost policy;Requirements review reports)

Berkeley Lab operates, DOE sets the mission

ESnet has no corporate hierarchy like a telecoms company. Authority is distributed across a chain of federal programmes. The DOE Office of Science sets the mission; Advanced Scientific Computing Research provides the primary programme oversight and budget management. Berkeley Lab's Scientific Networking Division designs, operates, and develops the facility, while the University of California manages Berkeley Lab under contract with the DOE. Connected institutions participate through site coordinators and the ESnet Site Coordinators Committee.

Inder Monga is ESnet's Executive Director and leads Berkeley Lab's Scientific Networking Division. Publicly available leadership information lists Chin Guok as Chief Technology Officer and head of Planning and Innovation, Adam Slagell as Chief Security Officer, Jon‑Paul Herron as Director of Network Engineering, and Susan Lucas as Deputy of Business Operations. Beneath them sit functions covering optical engineering, routing, site engineering, network operations, software, measurement, orchestration, security, project management, and science engagement.

The breadth of this organisation corrects any simplistic view of a national network as a collection of fibre and routers. ESnet needs staff who can procure spectrum, run BGP, build telemetry, maintain open‑source software, investigate packet behaviour, plan facilities, manage security information, and translate science requirements into network design. ESnet is an operational organisation and a development organisation at the same time.

Public information contains an unresolved discrepancy in programme management. The current ESnet governance page lists Benjamin Brown as the ASCR programme manager, while recent requirements review materials list Carol Hawk as the ESnet programme manager. It is possible that public sources have not been updated simultaneously, that responsibilities have transitioned, or that roles are shared. Without additional official confirmation, neither name can be asserted as the sole incumbent.

The ESnet Site Coordinators Committee gives connected institutions a formal operational voice. Each site designates a person who approves requests affecting its own connection, conveys requirements, and participates in policy and planning discussions. This is institution‑level user governance, not direct voting by every researcher who uses the network.

This machinery creates a practical division of responsibilities. ESnet manages its own backbone and services. Connected sites manage campus equipment, data transfer nodes, local security, and power. ASCR manages programme‑level funding, and science programmes define the impact of delays or failures. International partners manage their own networks. For a complete workflow to function, these distinct decision‑making domains must be aligned. (ESnet Leadership;Governance)

ESnet4 separated general IP traffic from massive science flows

By the mid‑2000s, distributed experiments were beginning to change the nature of network traffic. The Large Hadron Collider would generate data at CERN and distribute it to laboratories and computing centres around the world. General internet connectivity was still needed, but a handful of science transfers could become large enough to dominate ordinary links. Handling both patterns in one homogeneous network made capacity management and service guarantees difficult.

ESnet4 addressed this with a hybrid architecture. An IP core carried general science communication, while a separate Science Data Network provided high‑capacity optical circuits for large science flows. Major laboratories were connected with metro rings, and cooperation with Internet2 and international partners extended the network beyond DOE sites.

The distinction was not simply “slow traffic” versus “fast traffic”. General IP services require broad reach and flexible routing. In contrast, scheduled multi‑terabyte transfers can benefit from reserved Layer 2 paths with fixed start times, bandwidth, and endpoints. The Science Data Network recognised that a few predictable, high‑value flows need a different control model from ordinary packet forwarding.

OSCARS emerged from this environment. The On‑demand Secure Circuits and Advance Reservation System allowed authorised users or applications to request network resources for a specified period. The system explored paths, checked topology and policy constraints, reserved bandwidth and VLAN resources, pushed state to the equipment, and removed it at the end of the reservation. A task previously reliant on manual coordination by engineers became a service that software could request.

ESnet4 also showed that raw speed is not the only bottleneck. Even when a reserved circuit secures capacity across the wide area, the application stays slow if storage cannot read data fast enough, or if a local firewall drops packets. That experience fed directly into Science DMZ and end‑to‑end measurement.

During this period, a recurring design pattern for ESnet was set: a science problem first appears as a traffic requirement; the facility assembles physical capacity, control mechanisms, and operational practices around it; once the approach proves itself, the software and design guidance spread beyond the original experiment. (ESnet4 materials;OSCARS)

Science DMZ exposed the bottleneck at the organisational boundary

When the national backbone works well, a researcher's transfer performance can still be poor. The backbone is only one part of the path. General‑purpose firewalls, shared campus cores, ageing routers, under‑performing servers, packet loss, and storage constraints can all reduce the effective speed far below the capacity of a high‑capacity link.

Science DMZ is ESnet's answer to this end‑to‑end problem. It places high‑performance data transfer nodes on a dedicated path near the managed organisational boundary. Security policy is designed for specialised systems delivering a limited set of services, rather than forcing sustained science transfers through a multi‑purpose stateful firewall.

The name Science DMZ is sometimes misread as removing security. In fact it requires hardened hosts, router access control lists, monitoring, vulnerability management, restricted application exposure, and operational discipline. The design changes where and how control is applied, not whether security responsibilities exist.

Performance measurement is part of the architecture, not an optional dashboard. perfSONAR nodes measure throughput, packet loss, latency, and path changes under controlled conditions. When two sites disagree about a transfer failure, measurement can narrow the origin to the host, the local circuit, the regional network, the ESnet backbone, or the international partner. Without it, each operator reports its own segment as healthy while the researcher cannot move data.

Data transfer nodes also require specialist engineering. Network interfaces, CPU placement, memory, disk or parallel file systems, TCP buffers, and transfer software all affect effective throughput. ESnet's Fasterdata guidance and technical support turn these into reproducible practice. The approach has spread beyond DOE facilities, although not every implementation labelled Science DMZ is operated by ESnet.

Science DMZ gained wide acceptance because it turned a purchasing problem into a system‑design problem. Buying a faster circuit does not fix constraints elsewhere. The meaningful unit of performance is the complete workflow—from storage to storage, or from instrument to computing resource—and that requires measurement across each administrative domain so that responsibility becomes visible. (Science DMZ;Performance tools)

ESnet5 made nationwide 100 G a production reality

The next capacity shift was supported by the Advanced Networking Initiative and US economic stimulus funding. ESnet received $62 million through the American Recovery and Reinvestment Act to develop a 100 Gbps long‑haul prototype and migrate to ESnet5. The investment included not just new routers but also spectrum rights on national fibre, giving ESnet more direct control over wavelength deployment and renewal.

ESnet5 entered production at the end of 2012. At the time, the DOE and ESnet described it as the world's fastest science network. That historical claim should be understood in the context of a pioneering nationwide 100 G production deployment; it cannot be repeated unconditionally as a 2026 global ranking. Today's research networks report different metrics—individual interface speeds, aggregate backbone capacity, optical spectrum, laboratory trials—and the materials provided do not offer a neutral benchmark for comparison.

The effect of 100 G went beyond a ten‑fold increase in interface speed. Scientific data that would previously have occupied circuits for extended periods could now be planned as routine long‑haul transfers. Laboratories gained the option of centralising some computation or storage rather than replicating it at multiple sites. International collaborations received a more capable domestic US backbone. At the same time, the extra capacity exposed local constraints more quickly, increasing the value of Science DMZ and perfSONAR.

ESnet5 maintained the hybrid of packet services and dedicated science paths. Automation also grew in importance. At 100 G scale, the tasks of reserving capacity, diagnosing faults, and maintaining consistent service across the entire country could no longer rely solely on per‑device manual configuration.

The project demonstrated that public infrastructure investment can change scientific options before any single experiment fully occupies the capacity. Spare capacity serves future instruments and failure events; it is not waste in the same sense as unsold retail inventory. The ability to trial new workflows without waiting for the next construction cycle is also part of the value.

Nevertheless, this logic requires discipline. Capacity headlines must be tied to actual scientific use, and investments must be adaptable when demand forecasts change. ESnet's later work on telemetry, caching, and orchestration represents an attempt to extract more scientific value from installed capacity rather than always answering with physical augmentation. (ESnet5 announcement)

ESnet6 unified optics, packets, and software

The ESnet6 project started in 2017, went through roughly six years of design and construction, and was publicly launched on 11 October 2022. At launch Berkeley Lab announced over 46 Tbps of aggregate capacity on about 15,000 miles of dedicated infrastructure, with backbone links from 400 Gbps to 1 Tbps. The 2024 annual report, reflecting subsequent expansion, lists 57 Tbps and links up to 1.2 Tbps. These are figures from different moments, not contradictory definitions of a static network.

The physical layer comprises long‑haul fibre, optical line systems, amplifiers, spectrum rights, colocation facilities, and local connection points. The 2024 report counted 278 optical amplification sites. Coherent optical technology carries multiple high‑speed channels on a fibre pair; amplifiers and reconfigurable optical equipment maintain signals and steer wavelengths in the right direction. A single route on a map may be built from different forms—dark fibre, spectrum, lit service, partner contracts. The figure of about 15,000 miles does not imply that the DOE legally owns every cable.

Above the optical layer, routers and a service edge provide IP, private routing environments, Layer 2 services, peering, and cloud connectivity. The 2024 report listed 78 router sites. Each site connects at 10, 100, or 400 Gbps according to its requirements; backbone links aggregate multiple interfaces or wavelengths to build larger capacity.

What distinguishes ESnet6 from a mere capacity increase is the software layer. Automation maintains equipment and service state; programmable telemetry provides richer operational evidence. OSCARS sets up reserved paths; SENSE attempts application‑driven orchestration across multiple domains. High‑Touch uses programmable hardware to create fine‑grained visibility into packets and flows. Connections to APIs and testbeds move proven research results closer to production.

ESnet6 also emphasised resilience. Diverse paths, multiple site entrances, independent power feeds, alternative routers, and high‑availability service points reduce the chance that a single failure isolates a laboratory from the national system. However, each measure has a limited scope. Redundant backbones are useless if two local circuits share the same duct. Adding more routers does not create resilience if both depend on the same power supply.

ESnet6 is best understood as a layered science facility. Fibre creates physical reach; optical equipment creates capacity. Routers create packet and private services; software provides repeatability and programmability. Measurement verifies that promised paths actually work. Inter‑organisational agreements are necessary to cross boundaries that ESnet does not own. (ESnet6 launch;2024 annual report)

What 57 Tbps means—and what it does not

Network statistics tend to collapse different layers into a single figure. The 57 Tbps that ESnet cited for 2024 is an aggregate of design capacity distributed across the entire system. It does not mean a single physical link, throughput available to every laboratory, or traffic that is constantly flowing. The same report also lists backbone links of 400 Gbps to 1.2 Tbps, seven domestic US sites connected at 400 G or above, and 2.7 Tbps of trans‑Atlantic capacity. Each number answers a different question.

The report also states 78 router sites and 278 optical amplification sites on approximately 15,000 miles of infrastructure. The network connects all 17 DOE national laboratories, 28 DOE user facilities, 277 research and education, commercial and other network relationships in five countries, over 30,000 organisational users, and 137 staff and contractors spread across 24 US states. These are different populations; they should not be summed as a total customer count. Routers, facilities, partner networks, institutional users, and employees are separate metrics.

Site connections indicate capacity at one institutional boundary. Backbone links indicate capacity between network points. Optical systems carry multiple wavelengths, and logical services may use only a fraction of the physical layer. Aggregate capacity sums resources that cannot all be used simultaneously by the same source and destination. Traffic volume measures data moved over a period, not the peak rate at any instant.

ESnet carried 1.77 exabytes in 2024, a 4 % increase from 1.7 exabytes in 2023. That growth is lower than the roughly 55 % per year that ESnet shows as a long‑term average since 1989. A single year's result does not prove that science demand has stabilised. Large projects come online unevenly; software changes can reduce the amount transferred; one experiment or international event can alter the profile dramatically.

Traffic is also not uniform. The 2024 report states that Large Hadron Collider‑related traffic accounted for roughly half of the total. Fermilab was the largest external source at 136 PB; NERSC was the largest DOE internal destination at 75.3 PB; Oak Ridge was the largest DOE internal source at 58.3 PB. These numbers indicate dominant workloads but not a complete ranking of scientific value. Smaller transfers can be time‑critical or support unique instruments.

Capacity planning must consider average and peak traffic, per‑path utilisation, headroom for failures, site connection speeds, science‑imposed deadlines, and future requirements together. A network designed only for average utilisation may be starved by a supernova or a submarine cable break. A network designed only for the theoretical maximum event may build expensive, inflexible assets.

ESnet's engineering challenge is to place capacity where it can be reached by the necessary workflows, and to verify that the systems at both ends can actually use that capacity. For this reason, the 57 Tbps figure should be explained alongside Science DMZ, measurement, orchestration, and requirements reviews, not treated as a world ranking in isolation. (ESnet by the Numbers)

AS293 routes for science mission, not retail

ESnet's public Autonomous System Number is AS293. It exchanges routes with DOE sites, research and education networks, commercial networks, and paid transit providers. Routing policy determines which paths carry science traffic, how traffic fails over, and which external networks connect directly.

The peering policy is selective. Candidate networks must use BGP, keep Internet Routing Registry and PeeringDB records up to date, avoid RPKI‑invalid routes, and follow routing‑security practices aligned with MANRS. ESnet requires IPv6 or dual‑stack peering; new IPv4‑only relationships are not accepted. Direct private interconnections start at 100 G ports, with 400 G strongly recommended.

These conditions reflect both mission and operational cost. Every direct connection consumes a port, engineering time, and monitoring capacity. ESnet does not need to peer with every network merely because a public path exists. It selects relationships that improve scientific paths for the DOE, reduce transit dependency, and connect to major collaborators. For destinations not reachable efficiently through research networks or direct peering, ESnet uses paid transit from Hurricane Electric and Lumen.

ESnet publishes BGP communities that distinguish directly connected sites, research and education networks, and commercial networks. Connected institutions can use that information in their own policies, although ESnet cannot guarantee that downstream networks interpret the communities identically. Communities are descriptive metadata, not a promise of uniform routing decisions.

Routing security also has limits. Rejecting RPKI‑invalid routes prevents many instances of mis‑origination, but it does not stop all route leaks, compromised peers, routes leaked by a correct origin, or inaccurate ROAs. Hijack monitoring can spot unexpected origin changes, but detection may come after propagation. Blackholing protects other systems by discarding traffic to the target, but sacrifices reachability to that prefix.

AS293 carries public IP traffic, but its purpose is not to maximise customer count in a consumer transit market. The value lies in the quality of paths offered to the science community, resilience under failure, and trustworthy interconnection. (Peering policy)

Physical connections, private services, and paths to the cloud

ESnet services begin with physical connectivity. Eligible sites can connect at 10, 100, or 400 Gbps through a nearby or a remote Point of Presence. For facilities distant from dedicated infrastructure, a Service On‑Ramp may use dark fibre or a telecoms lit circuit. The access path is part of the science workflow, and a final segment provided by an external operator can become the principal bottleneck or failure point.

Layer 3 IP service offers IPv4 and IPv6 routed connectivity. Layer 3 VPNs create logically separated IP and BGP environments over a shared physical infrastructure for science programmes with multiple sites. Layer 2 VPNs provide point‑to‑point or multipoint Ethernet connectivity, either static or provisioned via OSCARS. These services allow experiments and facilities to connect without exposing every relationship to the public internet.

Private services do not imply physically fully isolated systems. Logical separation relies on router configuration, labels, routing instances, access controls, and operating procedures. A failure in the shared platform can affect multiple services; a configuration error can break intended separation. The value lies in managed separation and predictable connectivity, not in a claim that shared dependencies do not exist.

Cloud Connect extends ESnet paths to commercial clouds such as AWS, Microsoft Azure, Google Cloud, and Oracle. Sites can reach cloud connection points over dedicated Layer 2 or Layer 3 connections rather than through general internet routing. This improves path control and reduces exposure to unstable transit, but it does not eliminate cloud‑side security, provider outages, identity design, data egress charges, or regional availability.

The relationship with clouds signals a shift in ESnet's role. DOE science no longer resides only inside laboratories and federal supercomputers. Researchers can use commercial object storage, specialist services, and burst computing. ESnet must connect these resources without implying that the cloud is part of the federal network or that a dedicated circuit makes a cloud account secure.

Other services include secondary DNS, network time, route hijack monitoring, and blackholing. Even when optical capacity is healthy, science systems can become unusable because of name‑resolution failure, incorrect time, a route leak, or a denial‑of‑service attack.

The full service set is an operational arrangement spanning multiple network layers. Physical connections create paths; routing creates reachability. Private services create logical boundaries; cloud connectivity extends into commercial infrastructure. Security and ancillary services mitigate known failure modes. At both ends, users must maintain working endpoints and local networks. (Network services menu)

OSCARS turns capacity into a reservable resource

Ordinary internet traffic consumes capacity that is available when a packet arrives. That model suits routine communication, but some science transfers are scheduled, very large, and costly to delay. OSCARS enables authorised users or applications to reserve network resources for a specified period.

A reservation can include endpoints, a start time, an end time, bandwidth, VLAN information, path exclusions, and other constraints. The system examines the topology and existing reservations, selects a path that meets the conditions, pushes the required state into the network, and removes it when the reservation ends. It must prevent double‑booking of limited resources while handling scenarios where available topology changes because of maintenance or failures.

The important operational change is this: manual provisioning of a circuit might require several engineers and days of coordination; a programmatic reservation can be created in minutes and embedded in an experiment's workflow. The network becomes a resource that software can request, like storage or compute, instead of a fixed background facility.

OSCARS is an open‑source production system that has been deployed or evaluated in networks and testbeds beyond ESnet. However, published deployment information includes historical entries; not every listed organisation should be assumed still to be operating the same configuration.

Reservations do not guarantee application throughput. They can secure a certain amount of network capacity, but they cannot make storage read faster, repair packet loss on the campus network, or tune the operating system appropriately. If an application can only use a fraction of the reserved rate, diagnosis must return to the end‑to‑end path.

Reservations also raise resource‑allocation questions. High‑priority experiments benefit from guaranteed capacity, but idle reservations can reduce flexibility for others. Who may request a service, how contention is resolved, and whether reservations can be pre‑empted are facility‑governance questions even when the code is automated.

OSCARS is a clear example of ESnet's applied software becoming infrastructure. It turned a manual procedure into a repeatable service and, while maintaining human policies on authorisation, gave science applications a means to express network requirements. (OSCARS)

SENSE co‑ordinates networks and end‑systems together

A reserved network path solves only part of a distributed workflow. Storage, transfer services, and computing resources also have availability, capacity, and policy. SENSE—Software‑defined network for End‑to‑end Networked Science at the Exascale—aims to extend orchestration beyond a single network domain.

SENSE represents networks and end‑systems as discoverable resources. A science application can describe a goal, such as moving a dataset between facilities at a specified rate. SENSE gathers resource models from participating domains, negotiates resources, configures networks and transfer services, monitors telemetry, and releases resources after the workflow completes.

This is harder than provisioning a single circuit. Participating institutions keep their own authority and data models. One domain may advertise bandwidth, another storage endpoints or transfer services. They have different policies, credentials, and maintenance windows. SENSE must co‑ordinate without assuming that a single central controller can command every system.

During the 2024 Large Hadron Collider Data Challenge, a SENSE‑enabled workflow sustained 330 Gbps between Caltech and the University of California, San Diego. This is evidence that large‑scale, application‑driven orchestration worked in a specific operational workflow. It does not imply a production service available to every ESnet site. The 2024 report itself described SENSE as a research stage.

The difference between OSCARS and SENSE shows a changing unit of automation. OSCARS reserves network resources within a known service domain. SENSE attempts to negotiate a complete path across systems owned by different organisations. The former has clear production boundaries; the latter offers higher scientific value while carrying greater governance and integration risk.

For operators, the practical question is fault attribution. When an application states a goal and the workflow underperforms, the cause could lie in a resource model, the network, storage, credentials, the application, or inter‑domain timing. Orchestration must supply evidence that tells each operator which part of the negotiated state diverged from intent.

SENSE targets an infrastructure where applications request outcomes, not circuits. Its maturity should be judged not only by peak transfer speed but by repeatable, multi‑domain production use, clear support responsibilities, and the ability to recover from partial failure. (SENSE;2024 applied research)

EJFAT sends science events directly to remote computing

Traditional science data movement is often file‑centric. An experiment generates data; a local system writes it to files, stores it, and later transfers it to another facility for analysis. While established as an operating practice, this approach delays results and requires local storage and compute resources sized for collection peaks.

EJFAT—ESnet–Jefferson Lab FPGA Accelerated Transport—offers a different path. It delivers UDP‑encapsulated event data in real time from instruments to available compute workers, including remote supercomputing facilities. An FPGA‑based programmable SmartNIC reads a per‑event ID and sends all fragments belonging to the same event to a single worker. A control system adjusts the delivery target according to compute‑node availability.

Per‑event grouping is critical. A generic load balancer can spread packets across many servers, but in science processing, all fragments from one detected event must reach the same worker. EJFAT separates high‑speed forwarding from compute‑worker scheduling, allowing each to scale independently.

A demonstration in April 2024 streamed data at 100 Gbps from Jefferson Lab in Virginia to the Perlmutter supercomputer in California, processing it in real time without intermediate disk. ESnet reported that later tests used computing resources across multiple facilities and about 20,000 cores. This was a specific demonstration result; it does not guarantee that every science instrument can be deployed in the same configuration.

The Facility for Rare Isotope Beams example is a more recent operational illustration. In July 2026, ESnet reported that researchers streamed raw data at roughly 5–6 Gbps over EJFAT to a remote supercomputer. The 615 GB generated in 15 hours was processed in 20 minutes using machine‑learning inference on eight nodes of Perlmutter. The network speed was lower than the 100 G demonstration, but the scientific value lay in receiving results earlier during a live workflow.

Real‑time streaming also changes facility economics. Instead of building local computing resources sized for every peak, detectors can use shared supercomputing. Researchers can see results while precious beam time remains. On the other hand, network failure, compute‑allocation problems, or event‑delivery errors directly affect processing during an experiment, not just a later file transfer.

EJFAT is an advanced prototype or emerging platform, not a widely offered standard service. Production deployment would require instrument integration, security, FPGA supply, operational responsibilities, scheduling agreements, and clear behaviour under UDP packet loss. Its significance lies in showing that the wide‑area network can become part of the data‑acquisition loop, not just a post‑collection transport. (EJFAT;FRIB case study)

High‑Touch, perfSONAR, and iperf3 make the path observable

A high‑capacity network can fail in ways that aggregate utilisation graphs do not show. Short microbursts overflow queues; packets reorder; loss appears in one direction or only for some traffic; paths change between tests. Security‑critical flows can be buried inside statistical samples.

High‑Touch uses programmable hardware and software to collect packet and flow telemetry more detailed than conventional sampled records. In 2024 it was used to investigate Large Hadron Collider throughput, packet reordering on a Rubin Observatory path, and a security incident. The aim is diagnostic precision that can identify behaviour invisible in aggregate traffic.

Higher‑resolution evidence carries a cost. Packet and flow detail needs storage, processing capacity, access controls, and careful interpretation. It can reveal communicating endpoints, collaboration structures, and the timing of scientific activity. A system built for operational visibility becomes a sensitive data asset at the same time.

perfSONAR addresses a different layer. It is a distributed measurement infrastructure supported by a multi‑organisation collaboration, with ESnet as a founding and leading entity. In 2024, over 2,000 sites were reported globally, and ESnet operated more than 30. Controlled tests measure throughput, latency, loss, and path changes between participating nodes.

The distributed design is critical. When a researcher reports slow transfers, a properly designed perfSONAR test between nodes can show whether the underlying path can achieve the expected rate. Directional tests can reveal asymmetry; repeated measurements can indicate when degradation began. Traceroute can show path changes, although the observed path may not match the application's actual forwarding path exactly.

iperf3 provides active throughput testing with TCP, UDP, and SCTP, supporting IPv4 and IPv6. ESnet maintains this open‑source tool and reports tests exceeding 150 Gbps on 200 G paths under specific conditions. A synthetic test is not an application benchmark, but it can confirm that the host and network can carry traffic under controlled settings.

High‑Touch, perfSONAR, and iperf3 observe different realities. High‑Touch captures production flows in detail; perfSONAR measures paths across multiple domains periodically or on demand; iperf3 directly tests endpoint and network throughput. No single viewpoint gives a complete diagnosis, but together they reduce the risk of judging a distributed fault from a single operator's local dashboard. (Performance tools;2024 operational innovations)

The best capacity improvement may be not moving the data

Adding capacity is not the only way to improve a science network. When a particular dataset is repeatedly requested by researchers in the same region, moving every copy across the wide area consumes capacity and adds latency, even if the backbone is not congested.

ESnet's 2024 applied research report discussed five regional cache nodes used by science data communities. Across the survey, caches reduced wide‑area traffic by an average of 33 TB per day. Two‑thirds of requests were served locally, and the average cache hit rate was 94 %. The effect varied sharply by region, with reported wide‑area traffic reductions of 69.3 % in southern California, 48.4 % in Chicago, and 6.6 % in Boston.

The differences show that cache value depends on dataset popularity, regional users, storage capacity, eviction policy, and service configuration. Placed close to a community that repeatedly analyses the same data, a cache can cut large transfers. In an environment with low‑frequency, diverse data, it may consume storage without the same benefit.

Caching also changes the scope of control. The network operator becomes involved in data placement; the science community must decide which data can be replicated, how freshness is maintained, and who may access it. Storage failures or stale copies become operational problems adjacent to the network.

The most efficient bit may be the one that never enters the backbone. Optical upgrades increase supply; caching changes demand. Workflow scheduling can avoid peak hours; compression and local filtering can shrink data before transfer. Mature infrastructure planning evaluates these levers together, rather than measuring progress only by installed terabits.

The survey findings should be understood as specific to the environments studied. Not every science dataset can achieve a 94 % hit rate, nor can caches replace the new capacity needed for unique live streams. The important point is that data architecture and network architecture can be designed as one. (2024 applied research)

Trans‑Atlantic science requires physical diversity, not just capacity

ESnet's domestic infrastructure is inseparable from its international role. High‑energy physics spans facilities and computing centres on both sides of the Atlantic; the 2024 annual report stated that Large Hadron Collider‑related traffic accounted for roughly half of ESnet traffic. Every US router can be healthy, yet a submarine cable failure between Europe and North America can block principal scientific workloads.

During 2024 and early 2025, ESnet's published trans‑Atlantic capacity grew from about 700 Gbps to 2.7 Tbps. A key part of the plan was a 15‑year contract with Aqua Comms securing a quarter of a fibre pair on a route connecting New York, Dublin, and London. ESnet shares capacity and costs with GÉANT across multiple submarine cables. The engineering goal is to provide at least 3.2 Tbps over four physically diverse paths, with a design that can scale beyond 10 Tbps.

Capacity and diversity are separate problems. Two logically distinct services can share the same submarine cable, landing station, or terrestrial duct and fail together. Submarine cable repair requires locating the fault, securing permits, a cable ship, suitable weather, recovering and testing the damaged fibre—a process that can take weeks to months. ESnet plans for three simultaneous cable faults, indicating that a single backup path is considered limited public evidence.

Long‑term spectrum rights give more control over upgrades than repeatedly purchasing finished circuits. At the same time, they create long‑term commitments to specific cable systems, landing stations, and operating partners. Spectrum contracts do not eliminate marine risk; they let ESnet spread capacity and resilience across multiple systems while continuing to depend on cable owners, repair processes, and partner networks.

The word “dedicated” also needs care. ESnet manages dedicated rights and dedicated infrastructure on key routes, but the physical assets are not uniform. Domestic and international paths include dark fibre, spectrum, lit services, colocation, and partner capacity. A single line on a map does not reveal the ownership or repair authority for each segment.

The relationship with GÉANT shows that research networks can collaborate without being centralised. Both organisations share costs and can co‑ordinate science traffic, but each remains accountable to its own institution and entities. End‑to‑end performance also depends on national research networks, campus systems, and experimental facilities outside both backbones. (Trans‑Atlantic milestone;2024 annual report)

Science demand combines sustained high throughput, rare bursts, and strict deadlines

The Large Hadron Collider generates sustained high throughput. Its distributed computing model moves experiment data among CERN, Fermilab, Brookhaven, universities, and other computing centres, creating continuous international traffic. The planned High‑Luminosity operation is expected to increase detector data, simulation, and replication. The network will need to carry routine enormous flows while offering path diversity to route around submarine cable or facility failures.

The Vera C. Rubin Observatory illustrates time constraints. The telescope in Chile is designed to produce an image of roughly 13 GB every 30 seconds. Data must be processed swiftly at SLAC, and alerts sent to other observatories for transient phenomena. ESnet has stated a goal of moving each image across roughly 12,000 miles in under seven seconds. The path includes South American and international research networks; the outcome depends on the engineering of multiple organisations, not ESnet alone.

The Deep Underground Neutrino Experiment (DUNE) anticipates extreme bursts. In the event of a supernova, up to 600 TB may need to be moved in 100 seconds, with roughly 900 PB generated over the 20‑year programme. This is a future design requirement, not current day‑to‑day traffic. But astronomical events cannot be deferred until capacity becomes available. A network designed only for average utilisation may be starved during its most scientifically valuable moments.

Fusion research combines international distances with long‑term development. In May 2026, 176 TB of ITER data were transferred at roughly 80 Gbps from Marseille to the DIII‑D facility at General Atomics in San Diego. ITER is still under construction; this was a preparatory test, not routine full‑scale operation. Requirements documents anticipate about 2 PB per day and a minimum of 200 Gbps for some future workflows.

The Facility for Rare Isotope Beams and Jefferson Lab show another pattern. Experiments may need remote analysis while data is arriving. EJFAT can send event streams to computing resources at NERSC or Oak Ridge without waiting for files to be completed and copied. Light sources, electron microscopes, and other instruments face similar decisions about local filtering, remote AI inference, and the speed at which results return to instrument operators.

Climate and earth science add further demand. Large‑scale simulations and observational data move between computing facilities and storage, while field sensors may be located where ordinary fibre does not reach. No single headline speed can represent sustained high throughput, rare bursts, low‑latency feedback, international collaboration, and remote data acquisition at the same time. (Rubin Observatory case study;DUNE case study;Requirements reviews)

Requirements reviews convert science plans into network design

ESnet cannot wait for an experiment to produce its first data before procuring fibre, routers, and trans‑Atlantic capacity. Long‑term contracts, equipment purchases, site building, and software development take years. Therefore ESnet conducts requirements reviews with DOE science programmes, looking five to ten years ahead.

Researchers and facility teams describe instruments, projected data volumes, storage locations, computing destinations, time constraints, international collaborators, cloud usage, and resilience requirements. Networking and computing specialists examine the complete workflow. A request for a larger link may, on inspection, reveal issues with local storage, path diversity, security design, or transfer software.

Requirements reviews are a co‑design exercise. The science programme explains what it intends to achieve; ESnet translates that into infrastructure dependencies and verifies that the proposed path can work. The results influence site connection capacity, new routes, undersea capacity procurement, Science DMZ deployment, orchestration research, staffing, and the timing of the next‑generation network.

The large High‑Energy Physics review completed in 2025 and published in January 2026 involved 127 people and 14 case studies compiled into some 400 pages. It is important that not every prediction will be correct. Instrument schedules slip, software becomes more efficient, and new workloads appear. The value is that researchers, network engineers, computing centres, and funders share the same assumptions and have a record that can be updated deliberately.

Science demand does not always grow smoothly. A supernova may not occur during an experimental period, while AI workloads can expand faster than a formal review cycle. Average traffic growth is useful for budget planning but does not substitute for case‑by‑case planning. The low 4 % growth figure for 2024 alone should not determine multi‑decade infrastructure decisions.

Requirements reviews also distribute accountability. Laboratories should not assume that a national backbone will automatically fix inadequate campus paths. ESnet should not assume that building capacity will compel science programmes to adapt. Writing the dependencies into a shared plan makes potential future disputes concrete. (Requirements review reports)

Wireless Edge and QUANT‑NET test the boundaries of the facility

Nationwide fibre networks suit fixed‑site connectivity, but scientific instruments are not always in such locations. Geothermal fields, environmental observatories, and time‑limited survey sites may lack telecoms connectivity, stable power, or economic fibre paths. ESnet's Wireless Edge explores how private cellular, Wi‑Fi, directional radio, and satellite systems can extend science workflows into such places.

A geothermal observation in Nevada, published in June 2026, combined private 4G using Citizens Broadband Radio Service spectrum, long‑distance Wi‑Fi HaLow, conventional Wi‑Fi, Starlink backhaul, directional radio, and portable, self‑powered towers. This was a design tailored to one set of field conditions; it is not an ESnet nationwide wireless service or a substitute for long‑haul fibre. Terrain, weather, spectrum, power, and satellite operators all become part of the service path.

The important organisational point is that ESnet did not end its responsibility at the nearest fibre point. The science engagement function treated data acquisition, backhaul, and wide‑area transport as a single workflow. Because remote research sites differ physically, a field‑by‑field adaptation can be more valuable than a uniform product.

QUANT‑NET explores a different boundary. The ASCR‑funded project is building a three‑node quantum network testbed connecting two Berkeley Lab sites and the University of California, Berkeley over roughly 5 km. Research targets include ion traps, photonic coupling, entanglement swapping, Bell‑state measurement, time‑sensitive control, and modular software.

The connection to ESnet lies mainly in control and orchestration. Quantum experiments require classical communication, precise timing, and co‑ordination across devices that are difficult to operate. Open‑source control software can improve experimental reproducibility and reduce manual tuning. The testbed is not a production quantum internet, nor does it carry ordinary large‑scale science data through entanglement.

Wireless Edge and QUANT‑NET should be evaluated by what they teach and what can be reused. Neither signals a wireless access service or a quantum link for every ESnet user. They broaden ESnet's research function in anticipation that future science networks may need new physical media and control methods. (Wireless Edge;Quantum networking research)

American Science Cloud and ESnet7 shift planning units from sites to workflows

Integrated Research Infrastructure and the American Science Cloud signal a change in how the DOE thinks about facilities. Instead of treating scientific instruments, networks, data stores, and supercomputers as separate services, the aim is to make them operate as a federated environment supporting science data, AI, and high‑performance computing.

ESnet provides the connectivity and orchestration layer. Data moves from instruments to storage and accelerators; users and services need managed access. Workflows must discover what resources are available; telemetry must indicate whether science objectives are being met. EJFAT, SENSE, OSCARS, Cloud Connect, and High‑Touch address parts of this problem, but none of them alone constitutes the full American Science Cloud.

At Confab26, Inder Monga was introduced as Project Deputy for the American Science Cloud, and the programme included demonstrations and next‑generation network planning. This is evidence of institutional work in progress, not proof that a nationwide science cloud was complete by August 2026. Access policies, a catalogue of connected resources, production support, and governance were still under development.

ESnet7 appears in the same planning environment. The name is real; sessions have covered telemetry, deep packet inspection, data movement, AI, and quantum networking. But the current production generation is ESnet6. The materials provided do not contain a final architecture, project baseline, vendor selection, full budget, or launch date.

This distinction matters. Network generation names influence research priorities, vendor relationships, and capital requests long before equipment is deployed. A planning programme should not be reported as a finished backbone. The most reasonable understanding is that ESnet is using operational experience from ESnet6 and current workflow research to decide what functions a seventh generation needs.

Speed will remain important, but programmability and intelligence may become equally so. A faster circuit does not tell an application where available compute resources are, does not isolate microbursts, does not schedule detector streams, and does not prove that multi‑domain services were released correctly. The strategic question for ESnet7 is how far the network should understand and co‑ordinate scientific workflows without becoming an unmanageable central controller. (Confab26;2024 applied research)

Federal funding supports the facility; it is not corporate revenue

ESnet does not operate its backbone by selling bandwidth to general consumers. Its primary funding comes from the High‑Performance Network Facilities and Testbeds budget line within ASCR. The FY2027 request was $103 million, exceeding the $97.261 million shown as the prior year enacted level.

This number is the most useful public indicator of programme scale, but it is not an ESnet profit‑and‑loss statement. The budget line includes network‑related testbeds, software maintenance, renewals, and research activities. It does not show the annual operating cost of the production backbone alone, capital expenditure per network generation, or revenue attributable to each site connection.

Public funding enables investment before commercial demand emerges. It allows ESnet to secure long‑term spectrum rights, maintain open‑source tools, support experiments whose traffic is modest but scientifically valuable, and build path diversity beyond current utilisation. On the other hand, this model depends on congressional appropriations, DOE priorities, and the Berkeley Lab management contract.

Site connections are not always free. ESnet's Site User Cost Policy may charge institutional users for connection‑related costs depending on sponsorship, eligibility, and additional requirements. The unit of relationship is the site, not the individual researcher. Individual end users are not separately billed or registered by ESnet.

Without independent financial statements, a full financial analysis is limited. Public information does not show dependence on all principal suppliers, payroll costs, depreciation, asset registers by route, or annual capital expenditure. Calling $103 million “ESnet revenue” is inaccurate. It is more accurate to say that a federal programme funds a broad activity line that includes the production facility, renewal, software maintenance, and testbeds.

The funding structure also influences strategic choices. A commercial network can withdraw from routes that cannot recover costs; a public science facility must weigh national mission, scientific opportunity, long‑term capacity, and budget constraints simultaneously. Its funding allows investment without immediate financial return, but that requires transparent prioritisation and evidence from requirements reviews. (ASCR FY2027 budget request;ESCC)

Availability, routing security, and telemetry create their own responsibilities

ESnet's 2024 report showed important availability results. All ten DOE Office of Science sites included in the tracked metric achieved 100 % service availability excluding planned maintenance. It was the first year in which every site simultaneously exceeded the 99.9 % requirement. The annual report also gave a 99.99 % figure for a wider scope. Both numbers are useful but have different coverage; they do not mean that every service, institution, and international path experienced zero outages.

The Site Resilience Program examines the physical entrances, routers, power, and failure domains of connected facilities. A national backbone can be redundant, but a single laboratory that depends on one local duct or one piece of equipment remains vulnerable. Resilience does not end with a backbone map; it must reach the building and the data‑transfer systems.

Routing policy provides another layer of control. ESnet rejects RPKI‑invalid routes in peering, requires up‑to‑date routing records, and offers route hijack monitoring and blackholing. These measures reduce known risks but do not make inter‑domain routing completely safe. Routes with a correct origin can still be affected by leaks or policy errors. Blackholing protects other systems by making a target destination unreachable.

Automation carries a similar trade‑off. Standardised configurations, inventories, and orchestration reduce manual errors and speed recovery. At the same time, an incorrect template, policy, or data record can propagate rapidly to many devices. Gradual rollout, independent verification, clear rollback or forward‑recovery methods, and limits on the privilege given to a single automation account are required.

Operational data demands the same care. ESnet's facility data policy states that router utilisation and NetFlow records may be kept indefinitely and replicated across east‑coast and west‑coast storage. Active perfSONAR measurements are kept for six months on a single disk without backup. Router utilisation rates and traceroutes are public; flow and security logs have restricted access.

This is not inconsistent with the statement that ESnet does not track individual end users as subscribers. A subscription and billing system is different from network metadata. Even without individual contracts, metadata can reveal endpoints, communicating institutions, timing, and traffic patterns. High‑precision telemetry improves diagnostics and security while also making data minimisation, access controls, periodic retention review, and responsible use more important.

The strongest resilience model is not a claim of complete, uninterrupted operation. It is the combination of physical diversity, site engineering, routing security, measurement, incident response, data governance, and clarity about the scope of each metric. (Availability milestone;Facility data policy;Peering policy)

What ESnet demonstrates

ESnet's history can be read as a succession of faster networks: specialist precursors, ESnet4, the nationwide 100 G ESnet5, and the multi‑terabit ESnet6. Yet that speed chronology alone misses the most important continuity. Each time that bandwidth alone could not solve a science problem, ESnet changed the surrounding architecture.

The Science Data Network separated massive flows from general communications; OSCARS made capacity reservable. Science DMZ redesigned the organisational boundary; perfSONAR and iperf3 made performance problems measurable. SENSE connected application intent across multiple administrative domains; EJFAT embedded the wide‑area path into the detector processing loop. Caching changed demand; trans‑Atlantic spectrum strengthened control over international capacity.

Organisational authority remains limited and distributed. DOE and ASCR set mission and funding; Berkeley Lab operates the facility; the University of California contractually manages the laboratory. Site coordinators approve changes affecting their own institutions; international partners manage their own networks. Cloud providers, telecoms carriers, submarine cable owners, and equipment vendors control other components on which ESnet depends.

This distribution is not an inconvenience that software can eliminate. It is the condition under which a science network operates. ESnet's practical achievement is not pretending that one institution owns the whole path, but creating enough co‑ordination for workflows to succeed.

Future developments must be assessed with maturity distinctions. ESnet7 was in planning; SENSE was described as research stage in the latest full report; EJFAT was an advanced prototype or emerging platform; QUANT‑NET was experimental; the American Science Cloud was under construction. Qualifying maturity is not a mark against technical value.

ESnet's lasting significance lies in the scientific user‑facility model. It treats a network as a scientific instrument, with requirements, operations, software, measurement, research, and public funding. A detector and a supercomputer thousands of kilometres apart produce little shared value unless the systems between them are designed as a single path.

Therefore, the important question is not whether ESnet is the fastest in any universal ranking. The materials provided contain no neutral ranking that compares all current metrics. A stronger, practical conclusion is that ESnet is, on public evidence, one of the world's leading science networks, and its greatest contribution has been to make distributed science operable end‑to‑end. (ESnet History;ESnet6 launch;2024 annual report)

Sources