Summary

  • The October 2021 interruption followed planned anti-DDoS maintenance, but the public record does not attribute the outage itself to a DDoS attack.
  • OVHcloud said a command concerning redistribution from BGP into OSPF caused the full Internet routing table to enter its IGP, overload router resources, and make IPv4 routing inoperable while IPv6 remained reachable.
  • Approval controls existed: OVHcloud said the change had passed CAB, MOP, and peer review, so accountability turns on semantic validation, blast-radius limits, live-state observation, and effective rollback rather than an alleged absence of process.
  • Restoration was staged: the configuration problem emerged around 09:20 CET, the faulty router was powered off at 10:18, first services returned at 10:20, and the technical crisis was marked ended at 10:57.
  • The evidence supports a routing-reachability failure with broad service effects, but it does not establish an exact customer count, data loss, physical destruction, a complete financial-loss total, or a proven copy-and-paste root cause.

Maintenance intent was not operational cause

On 13 October 2021, OVHcloud began planned work on production routing infrastructure. The stated purpose was to strengthen anti-DDoS protection during a period in which attacks had become more intense. That context matters, but it must be kept in the right place in the causal chain. It explains why the change was undertaken. It does not establish that hostile traffic caused the outage.

The distinction is more than editorial caution. Operators routinely change routing, filtering, capacity, and detection systems in response to threats. If every failure during security-motivated maintenance is described as an attack, responsibility shifts away from the design and execution of the change itself. The relevant question becomes whether an adversary was active, even when the documented failure was an internal control-plane event. Here, the cited sources describe a configuration command, route redistribution, resource exhaustion, convergence failure, and physical isolation.

They do not describe a DDoS overwhelming the network at the time of the interruption.

That makes this a change-control case. The maintenance had an intended security outcome, but the running network followed the state produced by the command. Intent did not limit the number of routes entering the internal protocol. The existence of an anti-DDoS purpose did not make the new state safe. Security work still has to satisfy ordinary continuity requirements: limited scope, predictable semantics, observable effects, a tested reversal path, and a means of containing a bad state before it spreads.

This boundary also prevents an easy but unsupported story in which an external event and an internal error blend into one cause. Threat pressure can create urgency. Urgency can influence scheduling and risk tolerance. But the public evidence does not show that attack traffic forced the specific failure. The supported account is narrower and more useful: during maintenance motivated by anti-DDoS improvements, the routing system accepted a state that it could not safely carry.

The control-plane path from BGP into OSPF

The technical mechanism is central. OVHcloud said the command concerned redistribution from Border Gateway Protocol into Open Shortest Path First. BGP is used to exchange reachability information across network boundaries and to select among paths based on policy. OSPF is an interior gateway protocol used inside an administrative network to distribute topology and reachability information. Redistribution can connect those domains, but it also creates a boundary at which route volume, attributes, policy, and failure behavior must be tightly controlled.

According to OVHcloud's incident account, a router did not interpret a command correctly. The consequence was not merely one incorrect destination or one failed session. The entire Internet routing table was announced into the company's internal routing system. The OSPF table filled, and the router's memory and processor were overloaded. A convergence loop between BGP and OSPF followed, leaving IPv4 routing inoperable.

That sequence explains why the incident cannot be reduced to a vague “network problem.” The failure crossed a protocol boundary. External reachability information entered an internal control plane at a scale the affected state could not sustain. The resulting load impaired the very routing functions needed to correct or route around the problem. Once a control plane is saturated and repeatedly reconverging, a logically simple repair command may no longer have a simple operational effect.

The phrase “full Internet routing table” is important without requiring an invented route count. Public routing tables are large and dynamic. Injecting that volume into an IGP can consume memory, processor time, link-state processing capacity, and convergence attention across participating equipment. The record does not establish the exact number of routes or routers affected, so those quantities should not be guessed. It does establish the qualitative boundary: the redistribution was far broader than the internal routing domain could safely absorb.

The mechanism also shows why command validation must be semantic. A parser may accept a line of configuration. A device may commit it. A peer reviewer may recognize every keyword. None of those facts proves that the resulting route set is bounded. For redistribution, the meaningful object of review is the predicted set of routes and attributes that will cross the boundary, together with the effect on every receiving control-plane component.

A safe change therefore needs more than confirmation that the syntax is valid. It needs a testable assertion such as: only this identified subset of prefixes can be exported from BGP into OSPF; anything outside the subset is rejected; route count and resource use remain below defined thresholds; and an unexpected increase stops the change before broad propagation. The OVHcloud incident demonstrates the consequence when the actual route set, rather than the approved intention, becomes the first decisive test.

A staged clock, not one convenient duration

The detailed operator timeline separates the event into stages. Planned work began at 09:05 CET. BGP isolation and configuration actions were recorded at 09:18. The network-configuration problem occurred at 09:20, and router performance trouble was detected and escalated at 09:21. By 09:30, the attempted rollback had failed and physical isolation had been selected as the recovery route.

The affected router was powered off at 10:18. First services returned at 10:20 as the network began to converge without it. OVHcloud marked the technical crisis ended at 10:57. An earlier same-day summary used coarser times for the intervention and isolation. The detailed post-incident account is the stronger basis for precision, while the earlier summary remains a reminder that public timings can become more exact as investigation proceeds.

These milestones describe different conditions. The start of the faulty state is not the same as detection. The decision to isolate is not the same as successful isolation. First service return is not complete stabilization. The end of the technical crisis is not proof that every customer-facing dependency recovered at that exact instant. Compressing those states into a single duration would hide the operational questions that matter most.

For accountability, the gaps are informative. Roughly a minute separated the recorded configuration problem from detection. Several minutes passed before rollback was judged unsuccessful and isolation selected. Much longer passed before the router was physically powered off. First restoration followed quickly after that action, while broader stabilization continued. The public record does not expose every command, access constraint, decision, or dependency inside those intervals. It nevertheless shows which recovery boundary proved decisive: removing the faulty router from the running topology allowed convergence to proceed.

Operators should publish incident clocks in the same staged form. A useful record distinguishes change start, first abnormal route count, first resource alarm, customer-impact onset, detection, escalation, rollback initiation, rollback verification, containment decision, physical or logical isolation, first successful end-to-end service test, and stabilization. Without those distinctions, a short “outage duration” can conceal a long period in which the network remained fragile or the outcome remained uncertain.

Review existed, yet containment failed

OVHcloud said the change had been prepared through its change advisory board, method of procedure, and peer-review process. That evidence rules out a simplistic claim that nobody reviewed the work or that no process existed. It creates a harder accountability question: why did those controls fail to prevent or contain a route redistribution with network-wide consequences?

Approval mechanisms often focus on whether required participants signed, whether a maintenance window exists, whether commands are listed, and whether a rollback section is present. Those checks create traceability, but traceability is not containment. A method of procedure can accurately record an unsafe action. A peer can confirm that a command matches the written plan while both plan and command omit a decisive guard. A change board can weigh business risk without seeing a machine-derived preview of the route set.

The incident suggests at least four separate review objects. The first is intent: what security or resilience outcome is being pursued? The second is configuration semantics: what state will each command create on the named device and software version? The third is propagation: which other routers and protocols can receive or amplify that state? The fourth is recovery: what remains available if the changed router, the management path, or the control plane becomes overloaded?

A review that covers only the first two can miss the blast radius. A review that covers propagation but assumes rollback will always execute can miss the recovery trap. A review that lists a rollback command without testing it under high route volume and processor stress can mistake a documented option for a usable control. The public account says rollback failed. It does not disclose enough to assign that failure to a particular person, product, or unreported constraint. It does show that the planned reversal did not restore the network when needed.

This is where running state outranks process artifacts. CAB approval is a record of a decision. The MOP is a record of intended action. Peer review is evidence that another person examined the plan. None of them carries packets or bounds route propagation. The network's operational truth was the set of routes installed, the resources consumed, the sessions converging, and the reachability customers experienced. Governance becomes effective only when it is tied to measurements that can stop or reverse the change.

The lesson is not to discard review. It is to make review falsifiable. Before execution, reviewers should be able to state what they expect to observe: a maximum route-count delta, permitted prefix families, stable CPU and memory ranges, no unexpected OSPF database growth, no feedback into BGP, and successful probes over independent paths. During execution, deviation from those expectations should automatically pause progression. After execution, approval should remain provisional until the observed route and service state matches the prediction.

IPv4 failed while IPv6 remained reachable

The public record preserves a significant protocol-family boundary. IPv4 routing became inoperable, while IPv6 remained reachable. That does not mean every IPv6-dependent service worked normally, nor does it establish that all IPv4 paths failed identically. It does show that the incident did not erase all connectivity in one undifferentiated way.

This difference matters for both diagnosis and resilience. If one address family remains reachable, operators may retain a path for monitoring, management, status publication, or selected customer communication. Whether that potential is realized depends on dependencies. A status page reachable only through affected IPv4 paths cannot communicate the distinction. A management tool that resolves through unavailable services may fail even if an underlying IPv6 path exists. A customer application may support IPv6 at the edge but still depend on an IPv4-only origin or third party.

The public reports observed that OVHcloud's own site and status access experienced failures. That is a continuity warning: the systems used to explain an outage can share its fate. An operator should know which incident functions remain reachable over an unaffected protocol family and should test them as complete services, not just as addresses that answer a low-level probe.

The split also narrows the technical narrative. It supports the operator's explanation of a routing-state failure concentrated in IPv4. It does not support claims of destroyed servers, lost customer data, or a physical data-centre event. Hosted systems can remain powered and intact while being unreachable because the network cannot advertise or forward the paths needed to reach them.

For customers, unreachable and destroyed can feel similar during the event: a site does not load, an API cannot be reached, and dependent transactions fail. For accountability, they are very different. Recovery from a route-control failure focuses on control-plane state, convergence, isolation, and reachability. Recovery from physical destruction or data loss involves different evidence, risks, and remedies. The October incident must not be blended with OVHcloud's separate March 2021 Strasbourg fire.

Broad effects without an invented total

OVHcloud described disruption across its network and an inability to process IPv4 traffic correctly for its own websites. Independent reporting observed customer servers and sites becoming unreachable, the company's public site returning errors, and status information being unavailable. Numerama reported thousands of affected sites and named examples from French public and commercial services.

Those observations establish material reachability impact. They do not establish a complete inventory. The cited sources do not provide an audited count of affected customers, services, routers, countries, transactions, or revenue. Named sites illustrate dependency; they should not be treated as the entire blast radius. “Thousands” is a reported scale description, not a substitute for a reconciled customer-impact ledger.

The same discipline applies to harm. When a hosted service becomes unreachable, users may miss sales, communications, publications, or administrative functions. The sources do not establish a complete financial-loss total, specific data loss, or physical damage from this incident. The responsible account says what was observed and keeps the rest open.

This uncertainty is itself operationally useful. Providers need impact systems that can connect control-plane symptoms to customer-facing outcomes without guessing. Route telemetry can show prefix withdrawal or instability. Service probes can show reachability. Customer support can show reports. Billing and dependency data can help estimate exposure. None alone provides the full count. A mature incident record explains how these evidence streams were reconciled and where uncertainty remains.

Public communication should also distinguish reachability from workload health. A server may be operating normally behind an unavailable route. Conversely, a route may recover while an application remains impaired because sessions, caches, queues, or dependencies need time to settle. The staged restoration in this incident makes that distinction especially important. First services returned at 10:20, but the technical crisis was not marked ended until 10:57.

Rollback was written; isolation restored control

The attempted software rollback failed. The public record does not provide enough detail to say whether failure arose from overloaded resources, command behavior, access constraints, convergence dynamics, or another unreported factor. It does establish that the planned logical reversal was not sufficient and that engineers moved to physical isolation.

At 10:18, the faulty router was powered off. First services returned two minutes later as the network converged. That proximity does not prove every hidden causal detail, but it strongly identifies isolation as the action associated with restoration in the operator timeline. The recovery path depended on the ability to remove a participant whose state could not be corrected safely in place.

This creates a concrete control requirement. Critical routing changes need an isolation path that does not depend on the health of the affected control plane. That may include out-of-band management, remote power control, independent console access, pre-authorized peer shutdown, or on-site intervention. The appropriate combination depends on topology and risk, but it must remain usable when normal routing, CPU, memory, or management reachability is degraded.

Isolation also needs a rehearsed consequence model. Removing a router can shed bad state, but it can also remove capacity or connectivity. Operators need to know which peers will reconverge, how much spare capacity remains, whether route reflectors or interior-protocol neighbors will amplify churn, and which services will be temporarily sacrificed. A containment action should be designed before the incident, not improvised after rollback fails.

The distinction between rollback and restoration is essential. A rollback command aims to return configuration to a prior state. Restoration means the network again delivers stable, correct service. A command can report success while stale routes or overloaded processes remain. A configuration can be reverted while convergence continues. Conversely, physical isolation can restore service without yet explaining why the original rollback failed. Each claim needs its own evidence.

What this incident asks operators to prove

The central accountability test is not whether the maintenance had paperwork. It is whether the controls could prove and preserve accurate routing state while the change ran. That test begins before the maintenance window with a model of permitted redistribution. The expected prefixes, route counts, attributes, filters, and receiving domains should be explicit and machine-checkable.

The next test is containment. A bad redistribution should meet a hard boundary before it can fill an IGP. Prefix limits, route policies, protocol separation, staged activation, shadow validation, and narrow canaries can reduce exposure. No single mechanism is universal, and the public sources do not reveal the exact protections then deployed. The incident demonstrates the need for independently enforced limits rather than confidence that the planned command is correct.

The third test is observability. The system should expose route-count deltas, OSPF database growth, BGP and OSPF convergence activity, processor and memory pressure, adjacency changes, update rates, and end-to-end reachability by protocol family. Alarms must be connected to stop authority. Detecting a performance problem is valuable; preventing further propagation can be more valuable.

The fourth test is recovery under stress. Rollback must be exercised when control-plane resources are constrained and when the changed device is no longer a reliable execution environment. Isolation must be possible over an independent path. Restoration criteria must include stable routing and customer-visible probes, not only a returned command prompt.

The fifth test is public evidence. A useful post-incident account identifies intended state, observed state, containment decisions, staged restoration, and known unknowns. OVHcloud's detailed timeline and mechanism provide more accountability than a generic apology would. The remaining question is whether the lessons were converted into durable controls and whether those controls have since been tested under realistic failure conditions. The available record does not establish their current effectiveness.

What the record does not settle

The sources do not identify the device model, software version, vendor, exact command, parser behavior, route count, or full topology. They do not assign responsibility to a named engineer or prove what any individual knew at a particular moment. Those gaps prevent a fair claim about personal fault or a vendor-specific defect.

Some reporting discussed a deleted leadership message and suggested a copy-and-paste error. The later official incident account does not establish copying and pasting as the final root cause. The supported statement is that a router did not interpret a command correctly and that the resulting redistribution caused the routing failure. Anything more specific should remain attributed and provisional unless stronger evidence appears.

The record also does not prove customer data loss, destruction of servers, fire, or physical damage. It does not provide a complete economic-loss figure or exact affected-customer count. It does not show that DDoS traffic caused the interruption. It does not show that the change lacked CAB, MOP, or peer review.

These limits sharpen rather than weaken the conclusion. A reviewed, planned security-related change produced a control-plane state that overwhelmed routing, disrupted IPv4 reachability, resisted software rollback, and required physical isolation. The accountable response is to demonstrate semantic route validation, enforced redistribution bounds, independent management access, effective stop authority, and service-based restoration evidence. Those are claims about the running network, where continuity is actually won or lost.

Sources