The region customers can leave but not always escape: AWS US-East-1 and the price of resilience

Summary

• The accountability paradox is not whether AWS offers more than one region. It does. The test is whether a customer can use that diversity during a provider incident without first calling impaired control planes, identity paths, DNS management APIs, monitoring systems, or support channels. A second region that exists but is unprovisioned, unauthenticated, unobservable, or unreachable without a live configuration change is inventory, not operational resilience.
• On 19-20 October 2025, a latent race condition in DynamoDB's automated DNS management system caused the public regional endpoint in US-East-1 to lose all IP addresses. Three independently running DNS Enactors across three Availability Zones did not contain the failure because they shared one regional plan sequence and one cleanup logic. The automation entered an inconsistent state and required manual repair.
• Restoring DynamoDB endpoint resolution after about three hours did not restore the region. EC2's host-management system had lost leases and entered congestive collapse; network-state propagation accumulated a backlog; Network Load Balancer health checks removed healthy capacity whose configuration had not arrived; and dependent services throttled, failed, or drained queues for many more hours. AWS described three main periods of customer impact, with some Redshift recovery continuing into 21 October.
• The event did not mean every US-East-1 machine failed. Existing EC2 instances remained healthy, and some statically provisioned data planes continued serving. The failure instead impaired the ability to find DynamoDB, launch or network new capacity, process events, authenticate some requests, replace unhealthy components, and operate support and contact-center functions. That distinction explains both why some customers survived and why conventional auto-scaling designs failed later under daytime load.
• Public-sector records make the continuity consequence concrete without supporting claims of a universal government outage. NOAA said virtually all NESDIS products were affected and delayed rather than lost. The USPTO reported intermittent Patent Center interruptions and directed filers to alternate methods. A NASA science platform warned that notebook allocation could time out. Each case shows a different continuity requirement: preserve time-sensitive products, preserve legal filing routes, or preserve access to replacement compute.
• AWS controls the managed-service internals, global-service architecture, recovery algorithms, status publishing, and remediation evidence. Customers and downstream software providers control workload placement, pre-provisioning, dependency mapping, degraded modes, independent monitoring, and continuity procedures. Public bodies also control mission classification, procurement requirements, and non-digital fallback. Shared responsibility is not equal responsibility: it follows who could change the failed capability before the event.

A regional product with a non-regional escape problem

The cloud sales proposition is built around selectable failure domains. A customer can distribute an application across Availability Zones inside a region, replicate data to another region, and pay for a warm or active copy elsewhere. In theory, that makes resilience a purchasable property. In practice, the purchase only becomes real when the customer can exercise it while the primary environment is impaired.

That is where the control-plane paradox begins. An already running server can keep processing while the API used to describe or replace it is unavailable. A DNS record can keep answering while the API used to change it is down. A replica in another region can be healthy while the application still sends every request to the failed regional endpoint. A support service can fail over to another region and still reject users because an account-metadata dependency returns an authoritative-looking but invalid answer.

AWS's own Fault Isolation Boundaries guidance on global services is unusually explicit about this. In the standard commercial partition, the IAM, AWS Organizations, Account Management, Route 53 Public DNS, CloudFront, and several related control planes are hosted in a single region, often US-East-1. Their data planes may be distributed globally, and that separation can preserve established service. But the guidance tells customers not to rely on those control planes during recovery and lists operations in otherwise regional services that still depend on Route 53 or other single-region control functions.

The right accountability question is therefore not, "Did the customer buy a second region?" It is: Could the customer enter, observe, authorize, route, and operate the second region using paths that were already live and did not require the impaired authority?

That test is stricter than architecture diagrams. It asks whether capacity was pre-provisioned; whether data was current enough; whether credentials and trust policies worked; whether the failover control was a data-plane action; whether staff had independent communications; whether status evidence came from outside the provider; and whether the public service behind the system could tolerate the transition. It also asks whether AWS had kept its own recovery and customer-information systems outside the failure it was trying to explain.

October 2025 supplied a detailed answer. Some parts worked exactly as static-stability theory predicts. Existing EC2 instances remained available. DynamoDB Global Tables replicas in other regions could be addressed directly. Other parts exposed the price of hidden control dependencies: the regional database endpoint vanished, host leases expired, capacity could not be launched, new instances lacked network state, health checks withdrew usable load-balancer capacity, and support access was blocked despite regional failover.

The October 2025 clock had three outages inside it

AWS's post-event summary dates the event from 11:48pm Pacific time on 19 October to 2:20pm on 20 October and separates three periods: DynamoDB API errors, EC2 launch and connectivity failures, and Network Load Balancer connection errors. That is more accurate than assigning the event one start and one finish. Different services, control paths, and customer backlogs recovered on different clocks.

Pacific time	Event	Accountability significance
19 Oct, 11:48pm	An old DNS plan is applied after a newer plan; cleanup deletes the now-active old plan, removing all IP addresses from DynamoDB's regional endpoint.	Redundant workers share a plan-ordering defect and create one invalid regional answer. Automation cannot self-repair.
20 Oct, 12:38am	Engineers identify DynamoDB DNS state as the source.	Detection and diagnosis are relatively fast, but identification does not restore authoritative state.
1:15am	Temporary measures allow some internal services to reach DynamoDB and restore key internal tools.	Recovery first requires repairing the provider's ability to operate itself.
2:25am	DNS information is restored; cached answers expire through about 2:40am.	The trigger is mitigated after roughly three hours, but dependent state has already decayed.
2:32am	DynamoDB Global Tables replicas are reported caught up.	Cross-region replicas survived as an available target, though replication lag and customer routing still had to be managed.
4:14am	After several attempted mitigations, engineers throttle incoming work and selectively restart EC2 DropletWorkflow Manager hosts.	The host-management fleet has entered congestive collapse, and AWS says no established operational recovery procedure covered this state.
5:28am	EC2 host leases are re-established and some launches succeed under throttling.	Capacity returns gradually; an API success does not yet mean a usable networked instance.
5:30am onward	Some Network Load Balancers experience connection errors.	A later recovery phase creates a new data-plane consequence for previously healthy endpoints.
6:21am	EC2 Network Manager develops propagation delays while processing deferred network state.	The recovery queue becomes a separate bottleneck after host management improves.
6:52am	Monitoring detects alternating NLB health-check failures.	New instances without complete network state appear unhealthy, so protective automation removes capacity.
9:36am	AWS disables automatic NLB health-check failover.	Operators temporarily suspend a safety mechanism because its assumptions are false in the recovery state.
10:36am	Network configuration propagation returns to normal.	Newly launched instances can again become fully connected, but throttles remain.
11:23am	Engineers begin relaxing EC2 request throttles.	Recovery is admission-controlled to avoid recreating overload.
1:50pm	EC2 APIs and launches are reported normal.	The control plane recovers more than eleven hours after the regional endpoint first failed.
2:09pm	Automatic NLB DNS failover is re-enabled.	The normal protection system returns only after its inputs are trustworthy again.
2:20pm	AWS's detailed report marks the principal event ended.	This is a provider milestone, not proof that every service backlog or customer operation is reconciled.
3:01pm	Amazon's public update says all AWS services returned to normal operations.	A later public milestone reflects broader service restoration.
21 Oct, 4:05am	Operators finish restoring Redshift clusters trapped in replacement processes.	Some dependent resources remain impaired beyond the headline event window.

External measurements help test the boundary of the provider account. Cisco ThousandEyes' outage analysis observed early packet loss at the AWS edge near Ashburn, followed later by application timeouts and 503 responses as the failure moved through recovery phases. Those observations cannot reveal proprietary internals, but they support the conclusion that network reachability and application readiness recovered at different times.

AWS's public event history remains useful as the contemporaneous communication record. It should not be treated as a complete forensic report. A status history reports what the provider knew and chose to publish at a given moment; the post-event report adds mechanisms and later reconciliation.

The timeline also shows why "DNS was fixed" is an inadequate recovery statement. DNS repair restored a route to DynamoDB. It did not restore leases that had expired, network updates that had queued, instances that had not been created, event deliveries that had been throttled, contact-center sessions that had failed, or customer processes that had timed out. The recovery duration was the sum of dependent state transitions, not the duration of the first defect.

The trigger was DNS; the root was shared authority over state

The initiating mechanism was precise. DynamoDB maintains hundreds of thousands of DNS records for a large regional fleet of load balancers. A DNS Planner creates plans describing endpoints, load balancers, and weights. DNS Enactors, running independently in three Availability Zones, apply those plans through Route 53. Before acting, an Enactor checks that its plan is newer than the one already applied.

One Enactor became unusually slow and retried updates across several endpoints. While it progressed, the Planner generated newer plans and another Enactor rapidly applied one of them. The newer Enactor then began cleaning up old plans. At that moment, the delayed Enactor reached the main DynamoDB regional endpoint and applied its older plan. Its freshness check had happened much earlier and was now stale. Cleanup deleted the old plan that had just become active. All endpoint IP addresses disappeared, and the automation's state became inconsistent enough that subsequent plans could not be applied.

"A DNS error" describes the customer-visible trigger. It does not explain the control failure. The deeper conditions were:

• freshness was checked once at the start of a multi-endpoint operation rather than atomically at each decisive write;
• an older plan could overwrite a newer generation after prolonged delay;
• cleanup could delete a plan without proving that it was no longer active;
• independent workers in separate zones shared the same ordering and deletion assumptions;
• one regional plan simplified management for several endpoint types, increasing the authority attached to the plan;
• the invalid state was outside the automation's self-repair envelope and required a human to restore it.

This distinction matters because adding a fourth Enactor would not necessarily solve a protocol defect shared by all Enactors. Availability Zones separated processes and infrastructure; they did not create independent correctness. Redundancy multiplied actors executing the same unsafe transition.

AWS's remediation commitments follow that diagnosis. The company disabled the Planner and Enactor automation worldwide pending changes, committed to fix the race and prevent incorrect plans from being applied, proposed a velocity limit on how much capacity one NLB could remove during Availability Zone failover, added EC2 recovery testing, and promised queue-aware rate limiting for network-state propagation. Those are stronger measures than simply expanding capacity because they constrain authority and recovery speed.

They are still commitments in a provider-authored report. The public record reviewed here does not contain one independently audited closure register showing the deployment date, test coverage, failed test cases, remaining exceptions, and sustained performance of every action. Confidence in the causal account can be high while confidence in present remediation effectiveness remains lower.

A healthy server was not a recoverable service

AWS correctly emphasized that EC2 instances launched before the event remained healthy. That fact prevents the analysis from sliding into the inaccurate claim that US-East-1 physically went offline. It also exposes the exact risk customers were buying.

AWS defines control planes as the systems that create, describe, update, delete, and list resources, while data planes perform the service's primary work. Its control-plane and data-plane guidance explains why launching EC2 is a complex orchestration involving hosts, network interfaces, storage, credentials, and security configuration. The running instance is simpler. It can survive a period in which that orchestration cannot make a new one.

This is static stability in practical form. A service survives because it does not need to change. The weakness appears when demand rises, a host fails, a deployment replaces capacity, a certificate or secret requires renewal, a container exits, or an operator tries to fail over. Then the allegedly stable service calls the control plane at the least forgiving moment.

Buildkite's customer post-incident review illustrates the delayed failure. Its customer experience was initially stable. As US business traffic increased, EC2 launch failures prevented auto-scaling, shards exhausted their different capacity margins, and latency and errors rose hours after the AWS incident began. Buildkite preserved capacity by pausing deployments and later moved load to an otherwise unused shard. The decisive resilience asset was spare, already running compute, not the existence of an auto-scaling policy.

Postman documented a second form of coupling. Its outage review says critical flows were impaired, its AWS-hosted status page delayed communication, and its automated creation of internal incident channels depended on affected infrastructure. Postman accepted its own share of responsibility and described work on graceful degradation, multi-region capability, redundant communications, and eventually active-active operation across regions and providers. That is the right allocation: AWS owns the upstream failure; Postman owns the decision to let customer communication and coordination inherit it.

The October event therefore divides customer architectures into more useful categories than "single region" and "multi-region":

1. Running but change-dependent. Existing service continues until scaling, replacement, deployment, or credential refresh requires control activity.
2. Multi-zone but region-control-dependent. Physical zone failures are covered, while shared regional endpoints, control planes, and recovery systems remain common.
3. Multi-region but activation-dependent. Data and templates exist elsewhere, but failover requires provisioning, IAM changes, Route 53 changes, or unavailable operators.
4. Statically stable multi-region. Capacity, data paths, identities, health checks, and routing controls are already available, with failover using pre-positioned data-plane mechanisms.
5. Provider-diverse or manually continuous. A critical subset can run outside AWS, or the public function continues through a bounded non-digital process when cloud operation is not economical.

Only the fourth and fifth categories directly answer the control-plane paradox. The others may still be rational choices for lower-impact workloads, but they should not be presented as equivalent resilience.

Recovery automation became the second incident

Once DynamoDB DNS recovered, EC2's DropletWorkflow Manager attempted to re-establish leases with the physical hosts it managed. During the endpoint outage, those leases had timed out. A host without an active lease could not safely accept a new instance. The fleet's attempts took long enough that work expired and was queued again. AWS says the system entered congestive collapse and had no established procedure for that recovery state.

This is an important admission. The original race was a rare ordering failure. The prolonged EC2 impairment came from a foreseeable class of recovery load: many expired objects attempting to become current together. The exact scale may have been exceptional, but queues, timeouts, retries, and leases are ordinary distributed-system mechanisms. A recovery design must be tested at the fleet size it is expected to restore, not only for steady-state performance or incremental host loss.

The next phase made the dependency visible to running traffic. Network Manager had to propagate a backlog of configuration for new or changed instances. Some new instances existed before their network state was complete. NLB health checks saw failures, alternated between healthy and unhealthy, and withdrew nodes and targets from DNS. The checking system itself became loaded, and automatic Availability Zone failover removed capacity from multi-zone load balancers. AWS disabled the automatic protection at 9:36am to stop it acting on misleading evidence.

No single component behaved irrationally in isolation. Lease management refused unowned hosts. Network Manager queued configuration. Health checks removed unreachable targets. Availability Zone failover withdrew impaired capacity. The cascade emerged because each mechanism interpreted partial recovery as an ordinary local fault. Accountability sits in the cross-system design: recovery state must be represented well enough that one safety control does not punish another system for being temporarily incomplete.

The downstream effects were service-specific. Lambda throttled asynchronous and queue-driven work to protect synchronous invocation. ECS, EKS, and Fargate experienced launch and scaling failures. Amazon Connect saw failed inbound and outbound calls, busy tones, dead air, audio-message and call-routing failures, contact-center staff sign-in problems, and delayed reporting. STS experienced two periods of elevated errors. Redshift had both a regional dependency and a defect that sent an IAM group-resolution request to US-East-1 from all regions; local database users were unaffected by that particular cross-region failure.

That Redshift detail is especially revealing. A resource can be physically located outside US-East-1 and still call an endpoint there because of an implementation choice. Geographic deployment and dependency geography are not identical. Customers need the latter map, but only the provider can authoritatively disclose every internal service-to-service dependency.

Status and support are part of the safety system

During a cloud incident, customers need to decide whether they are seeing their own defect, an account-specific restriction, a regional event, or a global dependency. They need to know whether to fail over, freeze deployments, shed load, preserve queues, or invoke manual continuity. Status information is therefore an operational control, not public relations after the fact.

In October 2025, the AWS Support Center did fail over to another region. Yet an account-metadata subsystem returned responses that blocked legitimate users from viewing or updating cases. AWS had designed a bypass for failed responses; the dependency returned invalid responses instead. From 11:48pm to 2:40am, customers could not create, view, or update support cases through the console or API.

This is a classic semantic-failure problem. A dependency can be unavailable, slow, wrong, stale, or confidently wrong. Failover logic that handles only a timeout is not independent of a system that returns bad authority. Support continuity has to validate the meaning of account state, preserve a bounded last-known-good path, and offer a separately authenticated route for severe provider events.

The record shows improvement and recurrence at once. In the 7 December 2021 AWS service event, congestion between AWS's main and internal networks impaired monitoring, deployment tooling, control planes, the Support Contact Center, and the Service Health Dashboard's failover to a standby region. AWS promised a new support architecture active across multiple regions. The 2025 Support Center did move regions as designed, which is evidence of architectural progress. Its metadata dependency still prevented the service from fulfilling its purpose.

Customers also need to distinguish AWS's public status from personalized evidence. Current AWS Health Dashboard documentation says the unsigned public page shows public service events, while signed-in views provide account-specific events and resources. AWS recommends programmatic monitoring through EventBridge. Its guidance for public and account-specific health events recommends a backup rule so events can be delivered to an alternate region. AWS's regional event-rule guidance says global Health events such as IAM notifications require a rule in US-East-1, another reason to test rather than assume independence.

No organization should rely on one channel. A defensible arrangement combines provider public health, account-specific events delivered in more than one region, synthetic probes from another provider or on-premises network, application-level business metrics, and an incident page with independent hosting and identity. Contact lists, bridge details, and decision thresholds should be retrievable without the ordinary cloud account.

Public service impact was a continuity problem, not a website count

The October disruption reached public systems in ways that make simple outage totals misleading. The best evidence is service-specific.

The National Oceanic and Atmospheric Administration reported that its National Centers for Environmental Information cloud facilities began receiving low-ingest alarms around 06:57 UTC. A NOAA/NESDIS operational message said virtually all NESDIS products were affected and that data appeared delayed rather than lost. That is a materially different consequence from permanent data destruction. It can still be serious: environmental products are time-sensitive, and a six-hour delay can compress the time available to use observations in forecasts, planning, or downstream analysis.

The US Patent and Trademark Office said its Patent Center experienced intermittent interruptions and directed users unable to submit filings to alternate filing methods. That notice demonstrates a mature continuity principle: the legal or administrative function is filing, not availability of one web application. An alternative route preserves the function even when the primary interface is degraded. The record does not establish how many users invoked the alternative, whether every filing met its deadline, or why the incident was not marked resolved until 23 October; those questions should remain open.

NASA's Fornax science platform warned that starting a notebook server could time out while compute resources were allocated. That maps directly to the EC2 control-plane failure. Existing scientific data or notebooks need not have disappeared for research to stop; the inability to allocate a working environment is enough.

These records do not prove that all government services using AWS were affected, that emergency calls failed because of this event, or that any public-safety outcome occurred. They do show why procurement must look beyond where data is stored. A public service can depend on cloud control for product generation, filing, analytics, communications, or the compute needed to examine data.

CISA's August 2024 paper on public-safety communications dependencies warns that non-agency infrastructure can carry correlated continuity risk and recommends explicit redundancy, downtime procedures, backups, staffing, and support requirements. CISA's broader Infrastructure Dependency Primer asks whether a redundant provider is also shared by other systems and how long a workaround can be sustained. Those questions fit cloud architecture exactly.

A public body should classify continuity at the mission level:

• What result must still be produced if cloud control is unavailable for three, fifteen, or forty-eight hours?
• Which work can continue on already running capacity, and what demand margin exists?
• Which records may be delayed, and which legal or safety deadlines require an alternate path?
• Can staff authenticate, communicate, and publish public advice without the affected provider?
• Is a second region actually active, or must the organization provision it through the failed control plane?
• Can a manual process accept work, create a timestamped receipt, and reconcile it later without losing integrity?
• Does the contract provide technical evidence and support routes, not only credits after the event?

NIST's contingency-planning guidance remains relevant because it centers business impact, recovery priorities, alternate processing, and tested plans. The technology has changed; the duty to preserve the public function has not.

US-East-1 has a record, not one recurring bug

It would be misleading to describe every Northern Virginia event as the same control-plane defect. The mechanisms differ. The value of the record is that different triggers repeatedly exposed common questions about scope, internal dependency, recovery speed, and customer visibility.

Event	Trigger and mechanism	Dependency signal	Provider action disclosed
June 2012	A power event affected one Availability Zone; EC2 and EBS control planes across the region were also impaired.	Customers trying to replace zone capacity could not launch or attach resources elsewhere in the region during part of the event.	AWS described boot and recovery bottleneck work and changes to electrical transfer behavior.
February 2017	An authorized S3 operator entered an incorrect command input, removing more index and placement capacity than intended.	S3 APIs and AWS services depending on S3 failed; the status dashboard also lost some information because its administration console depended on S3.	AWS added command safeguards, reduced blast radius, and separated status administration dependencies.
November 2020	A modest Kinesis front-end capacity addition caused every server to exceed an operating-system thread limit.	Cognito, CloudWatch, Auto Scaling signals, Lambda, EventBridge, ECS, and EKS inherited the service failure; the normal status publishing tool depended on Cognito.	AWS committed to front-end cellularization, alarm and cold-start improvements, service partitioning, and regular training on a manual status tool.
December 2021	Automated scaling triggered a client connection surge that overwhelmed devices between AWS's internal and main networks; a latent backoff issue sustained congestion.	Monitoring, internal DNS, authorization, deployment, EC2 controls, support, and status failover shared the constrained path.	AWS disabled the triggering activity, added network protections, fixed client behavior, and promised active multi-region support architecture.
October 2025	A DNS-plan race removed the DynamoDB regional endpoint and left automation unable to self-repair.	DynamoDB dependency caused EC2 lease collapse, network backlog, NLB health-check instability, service throttling, support blockage, and cross-region Redshift IAM impact.	AWS disabled the automation globally, committed race and plan-safety fixes, NLB velocity limits, EC2 recovery tests, and queue-aware throttling.

The 2012 service summary is an early statement of the paradox: control planes are particularly important during an outage because that is when customers try to create or move resources. The 2017 S3 report showed an operational command with more authority than intended and restart durations that had not been experienced at the region's newer scale. The 2020 Kinesis report explicitly separated trigger from root cause, then described a many-hour controlled fleet restart and a status-tool dependency. The 2021 report exposed the provider's own reduced visibility and deployment impairment.

The recurring pattern is not negligence by a named operator, nor proof that AWS failed to learn. It is that scale changes the meaning of a safe operation; redundant components can share one logical failure; recovery demand can be larger than steady-state demand; and incident tools can share a fate with production. Each post-event action should therefore be tested against the next mechanism, not merely the last exact trigger.

There is evidence of learning. The 2025 Support Center had regional failover where the 2021 architecture had not protected case creation. DynamoDB used three independent DNS Enactors. EC2 preserved existing instances. Global Tables replicas remained addressable elsewhere. AWS published an unusually detailed causal report. The remaining issue is whether these controls fail safely when they receive slow, stale, or invalid state rather than a clean outage.

What resilience can actually be purchased

AWS's current Reliability Pillar tells customers to define recovery objectives, test disaster recovery, conduct game days, use static stability, and rely on data planes during recovery. The specific REL11-BP04 guidance identifies common anti-patterns: changing DNS records during an incident, scaling control-plane capacity because failover resources were under-provisioned, or depending on a chain of management APIs.

That guidance is technically sound. It also defines the bill. Static stability means paying for resources before they are needed, constraining deployments that would remove reserve capacity, keeping identities and data ready in another region, and operating a routing control whose data plane is already distributed. A customer that pays only for backups has bought data protection, not immediate service continuity. A customer with a pilot light has bought a faster rebuild, not immunity from control-plane failure. A customer with warm standby has bought capacity with a scaling dependency unless the standby can carry the promised load as provisioned.

AWS's disaster-recovery options describe backup-and-restore, pilot light, warm standby, and active-active models. They also advise using only data-plane operations for maximum resilience. The implication should be written into business cases: lower cost generally purchases more control-plane work at the moment of failure. The workload owner must decide whether that trade is acceptable, and executives must fund the answer that matches the impact tolerance they approved.

Multi-region is not an automatic verdict. DynamoDB Global Tables replicas outside US-East-1 were available during the 2025 event, but applications still needed a tested route to them, acceptable consistency behavior, adequate capacity, and a way to reconcile the recovered replica. Identity policies, KMS keys, secrets, certificates, container images, queues, observability, and third-party APIs all need the same review. A diagram with two database icons does not prove a complete business transaction can finish in both places.

Multi-cloud is not an automatic verdict either. Rebuilding every managed service against a second provider can introduce data inconsistency, operational errors, higher cost, and a platform that is exercised only during emergencies. The latest GAO review of federal cloud procurement found that agencies using multiple vendors also faced interoperability, staffing, tooling, and management challenges. The report records the more useful formulation offered by the Department of Defense: manage concentration risk and maintain architectural awareness and an exit strategy for mission-critical workloads, rather than treating all provider-specific capability as inherently wrong.

The practical answer is selective independence. Keep the most time-critical path statically stable across regions. Preserve a small read-only or intake mode when full service is too expensive. Use open data formats and tested exports for functions that may need another provider. Maintain a manual or offline process where legal and public obligations permit. Do not spend equal resilience money on a public information page, a benefits-payment instruction, an internal analytics job, and a safety dispatch function; their consequences differ.

Responsibility follows the capability that could have changed the outcome

"Shared responsibility" can become a fog if it is used to divide every failure evenly. AWS's own resiliency model assigns resilience of the cloud infrastructure and managed services to AWS, while customers choose configuration, placement, replication, backup, and workload architecture. The split is useful only when translated into concrete control capabilities.

Capability	Primary control holder	Accountability test after October 2025
DynamoDB DNS plan correctness	AWS	Can an old generation ever replace a new one, can cleanup delete active state, and can automation recover without an operator?
Cross-zone logical independence	AWS	Do redundant workers have independent failure assumptions, or only independent hosts?
EC2 host-lease recovery	AWS	Has full-fleet lease loss been tested, with queue limits, admission control, and a documented procedure?
Network-state backlog control	AWS	Does incoming work rate adapt to queue depth before timeouts and retries create collapse?
NLB health and failover velocity	AWS	Can health automation distinguish incomplete recovery from unhealthy capacity, and is removal rate bounded?
Internal service dependencies	AWS	Which regional and global operations call US-East-1, and are customers given enough information to design around them?
AWS Health and Support continuity	AWS	Can public updates, personalized events, and severe-case support function with primary identity, metadata, console, and regional paths impaired?
Region and service selection	Customer or downstream provider	Was the selected architecture proportionate to measured business impact and approved recovery objectives?
Pre-provisioned failover	Customer or downstream provider	Can traffic move without creating resources, changing global control planes, or obtaining unavailable authority?
Graceful degradation	Customer or downstream provider	Which transactions remain available, queued, read-only, or manually accepted when dependencies fail?
Independent detection and communication	Both, for their own operations	Does each party have external probes, an independent incident channel, and a status route that survives the primary provider?
Public-service continuity	Public authority and service supplier	Is the mission outcome preserved through alternate processing, deadlines, public guidance, staffing, and reconciliation?
Remediation assurance	AWS leadership, customer assurance functions, and where applicable public buyers	Are changes complete, tested at scale, sampled independently, and reported with exceptions rather than announced once?

This allocation avoids two errors. Customers cannot patch DynamoDB's DNS Enactor or create an EC2 recovery procedure inside AWS. AWS cannot decide whether a municipal filing portal merits active-active operation or whether a public agency has an acceptable manual intake process. A downstream SaaS company cannot blame AWS for hosting its own status page in the same failure domain, but it also cannot discover undisclosed provider internals through architecture discipline alone.

Responsibility also changes with service abstraction. A customer managing EC2 has more choices and more work. A customer buying DynamoDB delegates more platform operation and should expect AWS to manage the service's internal DNS and recovery correctly. The customer still controls replication and application failover. Managed service does not eliminate customer continuity duty; it narrows the internals the customer can control and increases the provider's duty to disclose usable failure boundaries.

Credits price a service metric, not the public consequence

Commercial accountability has a narrow contractual layer and a broader operational layer. The current DynamoDB SLA commits to monthly regional uptime levels, with a higher target for qualifying Global Tables use. The stated remedy is generally a service credit, subject to eligibility, calculations, exclusions, logs, and a claim filed through AWS Support.

That mechanism can enforce a measurable service commitment. It does not reimburse the whole cost of delayed environmental products, missed developer work, failed customer calls, lost downstream transactions, or public staff diverted to manual processing. Nor does an SLA determine whether a particular customer designed responsibly. Contract terms vary, and this analysis makes no finding that AWS owes any customer damages beyond a governing agreement.

The 2025 event exposes a procedural irony without proving a legal defect: the standard credit process uses the Support Center, while Support case functions were unavailable during the initial DynamoDB phase. Customers had later claim windows, so the temporary block did not necessarily prevent a claim. It does show why evidence needed for a credit should be collected outside the affected monitoring stack. CloudWatch, application logs, synthetic probes, provider events, customer transaction records, and manual incident notes should be retained independently.

The scale of the provider increases the governance expectation. Amazon's 2025 Form 10-K reports AWS net sales of $128.725 billion, up 20 percent, and separately acknowledges risks from system interruption and incomplete redundancy. Revenue is not proof of fault. It is evidence of capacity, reach, and an economic relationship in which reliability controls, transparent post-event reporting, and remediation assurance are core product obligations rather than charitable extras.

What evidence would change the conclusion

The current conclusion is that AWS provided substantial physical and data-plane resilience, but the October 2025 event exposed preventable common assumptions in regional DNS automation and insufficiently prepared recovery paths. Customers could purchase meaningful resilience, but only by pre-positioning service and avoiding control-plane dependencies that AWS itself documents. Some cross-region and support dependencies remained less independent than customers could reasonably infer from geography alone.

Several kinds of evidence would make that judgment more favorable:

• a dated public closure record for the DNS race fix, per-endpoint freshness enforcement, active-plan deletion protection, and automated recovery from inconsistent plan state;
• fault-injection results showing that delayed Enactors, stale generations, concurrent cleanup, partial Route 53 writes, and failed recovery cannot remove the regional endpoint;
• EC2 tests at realistic US-East-1 scale showing full lease reconstruction completes within bounded time without congestive collapse;
• queue-depth and admission-control evidence for Network Manager, including behavior under a regional backlog larger than October's;
• NLB tests proving velocity controls prevent false health transitions from withdrawing too much multi-zone capacity;
• a service-dependency inventory identifying global or single-region control operations that may affect workloads outside US-East-1, with changes tracked over time;
• demonstrated Support and AWS Health exercises in which identity, account metadata, the console, EventBridge delivery, and one region each fail independently;
• customer-facing recovery metrics that separate endpoint repair, control-plane repair, data-plane stability, backlog clearance, and resource reconciliation;
• independent assurance sampling the completion and durability of severe-incident corrective actions.

Evidence could also make the conclusion less favorable. Repetition of the same race after automation is re-enabled, another fleet recovery with no established procedure, undocumented cross-region calls to US-East-1, or status and support failure through the same metadata path would indicate that remediation treated symptoms rather than authority boundaries. A shorter outage alone would not settle the question; a lucky workload pattern can conceal an unsafe control.

Customer evidence matters too. A public agency that can show a full regional game day, independent communications, current data in a secondary region, pre-provisioned capacity, bounded manual service, and successful deadline reconciliation has converted cloud diversity into continuity. A customer that labels backups or templates as "multi-region" without a timed exercise has not.

AWS publishes a useful standard for when it will issue public Post-Event Summaries, including broad events involving significant control-plane failures or infrastructure impact. That archive is valuable. Stronger assurance would connect each severe report to a durable action register so customers and public buyers can distinguish an announced fix from a tested, completed, and sustained control.

The accountability finding

The October 2025 outage began with two pieces of automation disagreeing about time. One slowly applied an old plan; another applied a new plan and deleted the old one; together they erased the address of a regional database service. The long incident came from everything that had trusted that address and from the state that decayed while it was absent.

AWS owns that chain inside the cloud. It designed the plan protocol, the cleanup authority, the host leases, the recovery queues, the health checks, the service dependencies, and the support path. Its post-event report is technically specific, its immediate remediations match the mechanisms disclosed, and its preservation of existing EC2 instances validates the value of control-plane separation. The unresolved accountability issue is proof that the fixes work at regional scale and that hidden cross-region dependencies are made visible enough for customers to act.

Customers own a different chain. They decide whether peak demand can be served by running capacity, whether replicas can be reached without a new control action, whether status and incident coordination are independent, and whether the business or public function can degrade safely. Buildkite's exhausted shards and Postman's impaired status route were not causes of AWS's failure; they were customer-controlled multipliers of impact. NOAA's delayed products, USPTO's alternate filing path, and NASA's allocation warning show why the multiplier must be evaluated in operational terms, not by server count.

The central test can now be answered. Customers can buy regional resilience on AWS, but a second region is not sufficient evidence of the purchase. The usable product is a complete recovery path: already provisioned, already authorized, already observable, already routable, and rehearsed without the primary control plane. Where global controls or provider internals still converge on US-East-1, AWS must either remove the dependency, make its safe data-plane alternative clear, or state the limitation plainly.

Cloud concentration is often discussed as market share. The more immediate concentration is executable authority: one plan, one endpoint, one metadata answer, one health interpretation, or one support dependency that can make many redundant systems behave alike. Accountability begins by naming that authority before the next incident, then proving that a separate path still exists when it is wrong.