When us-east-1 Goes Dark: What One AWS Region Under Your Whole Stack Really Costs

Your Ring doorbell stopped answering. United's check-in screens went blank. Your bank's app spun, then failed, then told you to try later. None of those services look related — until you learn they all spent October 20, 2025 in the same place: AWS us-east-1.

At 03:11 ET that morning, AWS flagged elevated error rates in its Northern Virginia region. A race condition in DynamoDB's DNS automation had quietly broken the address book other AWS services use to find DynamoDB. Because DynamoDB stores session tokens, config, and metadata for dozens of sibling services, the failure cascaded across Lambda, ECS, EKS, Fargate, IAM, Amazon Connect, and more. DNS was mitigated at 02:24 PDT, but backlog drain and retry storms dragged the incident to roughly 15 hours — until about 15:01 PDT — before AWS declared full recovery.

That number — 15 hours — is the one to carry into every cost conversation below. Headlines quoted aggregate losses in the hundreds of millions. Those matter for insurance reports. They do not tell you what your app lost. This table does — plug in your own hourly number.

The worksheet, up front — find your row before you read the why

Formula: cost = (hourly_revenue + support_burn + SLA_credits) × duration + recovery_tail. Recovery tail is the extra hours after "all clear" spent draining queues, replaying webhooks, and answering tickets. Ignore it and you undercount by 20–40%.

Add $5k–$30k in support overtime and SLA credits for the mid-tier row, double that for the large row, and you land where Censinet did when it estimated $62,500 per hour for healthcare organizations alone during the same outage. CyberCube's $581 million in modelled insurance losses is the same event measured with a different denominator — the whole economy, not one tenant.

The rest of this post puts those numbers in context: why half the internet felt like it shared that one region, why you were in us-east-1 even though you never picked AWS, and what a fault domain you draw yourself actually has to get right to be cheaper — not just different.

The 15 Hours That Took Your Stack With It (and the Four Before It)​

The October 2025 outage was not a new species. It was the latest, longest instance of a pattern us-east-1 has repeated every 12–24 months.

Sources: AWS post-incident summary for Oct 2025 (DynamoDB DNS), Ookla/Catchpoint Downdetector analysis (17M reports vs 16M initially), GeekWire and TechCrunch contemporaneous coverage, Central Technology and Cato Networks impact roundups, StatusGator aggregations for July 2026, eG Innovations/Aviatrix timelines.

Three structural facts turn that pattern from "AWS had a bad day" into "you had a bad day":

1. us-east-1 is the default. The AWS docs still note it as the default region for many SDKs and services, and it is the oldest, largest region by service footprint. Teams that never set region end up there. So do many SaaS vendors.
2. DynamoDB sits underneath the control plane. Even if your workload stores nothing in DynamoDB, the AWS services you call do — for auth tokens, shard maps, quota state. When DynamoDB's endpoint cannot be resolved, IAM cannot verify you and Lambda cannot route you.
3. Blast radius is uncorrelated with product category. A smart lock, an airline DCS, and a hospital EHR have no reason to share a fault domain. In October 2025 they did, because their vendors shared us-east-1. TechTarget put it plainly: "organizations without direct AWS contracts experienced downtime because their SaaS vendors, payment processors and authentication services depended on the US-EAST-1 infrastructure."

That last point is why the worksheet above uses your hourly burn, not CyberCube's aggregate. Half the internet did not literally sit on us-east-1 — Cloudflare Radar and W3Techs put roughly a third of the top-million sites behind AWS, and us-east-1 holds the plurality but not the majority of AWS capacity. What was true is that enough of the middleware your app touches — auth, payments, email, error tracking, feature flags — had at least one critical dependency in us-east-1 that the median tenant was unknowingly single-region in exactly the way they would have rejected if it had been presented as an architecture choice.

Why You Were in us-east-1 Even Though You Never Chose AWS​

Do this 30-second check. You only need the answer to "where does the platform I pay actually run?"

Now count the second layer — the vendors those platforms call:

Each hop adds a shared fate you never drew. A single-app team in 2025 typically had 3–6 such hops before a request reached its own code. The question is not "do we use AWS?" It is "how many of the services we pay call AWS us-east-1 on our behalf, and do we know which ones?"

The Blast Radius Math: Shared Fate vs. Drawn Boundary​

Put the two worlds side by side with numbers, not adjectives.

The honest row is the monthly cost. Hyperscaler pricing wins at truly spiky, mostly-idle shapes — Azure Container Apps' 180,000 vCPU-second free grant, or Cloudflare Workers with zero egress for stateless bursts, can be cheaper than idle iron you own at 5% utilization. The flat-box pitch wins at steady utilization above ~50%, which is where the boring 24/7 API + worker + Postgres workload most teams actually run lives. Medium's 2026 Hetzner move post that measured an 89% cut moving a steady API from AWS to Hetzner is a single anecdote, but Ankra's six-cloud Kubernetes benchmark that same summer found the same directionality at equal instance class.

Use the worksheet again here. If your 15-hour loss under the large row is $1.5M, you can tolerate a lot of flat-box overhead before owning the boundary stops paying. If your 15-hour loss under the small row is $15k and you value never paging at 03:11 ET more than you value owning the metal, the hosted trade may still be right — but at least it is now a number, not a vibe.

What "Own the Fleet" Has to Get Right (and What It Doesn't Magically Fix)​

Owning the fault domain does not mean outages disappear. It means they become yours to prevent and to fix, with different failure modes. The ones that bite a Hetzner-backed Cluster API fleet are not price hikes — they are capacity and operations.

The three failure modes to wire for before you need them:

1. "No capacity in this location." Hetzner's 2026 status page has carried repeated limited availability notices as DRAM/NAND pressure constrains specific server lines. A MachineDeployment scaling in fsn1 can now fail with sold-out inventory, not a billing error. Mitigation is multi-location, multi-type fallback — never bet a fleet on one location and one SKU.

2. "The hardware failed, not the software." On bare metal you swap a disk via a support ticket; on Hetzner Cloud you delete the machine and let CAPH recreate. MTTR is hours, not minutes, and you own the spare-capacity math. Hold 20% headroom (e.g., 3 nodes where 2 carry peak) so one lost node is not an outage, or you have just traded a provider queue for a self-inflicted one.

3. "The account, not the app." Hetzner's abuse automation is famously sharp-edged — the "absurdly cheap, sole-owner, account-terminated-without-appeal" review is not fringe lore. Guard the fleet provider account as tier-zero infrastructure: 2FA, billing alerts, secondary contact, and a tested restore path to a second Hetzner (or Hetzner-adjacent) account and to another provider entirely.

A minimal CAPH shape that survives all three:

# 5 lines that matter — full module at github.com/bex-co/bex
failureDomains: [fsn1, nbg1, hel1]  # not one region
serverTypes: [cax11, cpx11, cx22]   # Arm + x86 fallback
replicas: 3                         # 20% headroom for N-1

That is the whole trick. Not a custom scheduler — just never let a single location string or a single server_type be load-bearing. The rest is standard Kubernetes: PodDisruptionBudget, TopologySpreadConstraints, and an off-cluster etcd backup you have actually restored.

The line to carry: self-hosting trades correlated multi-tenant outages and a bill that grows new line items (Fly.io's volume snapshots in January 2026, inter-region private networking at Machine rates in February 2026) for uncorrelated, operator-owned failures and a bill that is the same number every month. Whether that trade is good depends on your 15-hour row in the worksheet, not on a manifesto.

Designing for the Next us-east-1​

There will be a next one. The policy question is not how to prevent hyperscaler failures, but how to make their blast radius explicit before the status page turns red.

Three steps this week, not this quarter:

1. Map your inherited dependencies. List every vendor you pay that can put your app in us-east-1 on your behalf — hosting, auth, payments, DB control plane, email, error tracking, feature flags. For each, note its primary region and whether failover is automatic, manual, or nonexistent. You will find at least two you had not thought of.

2. Draw your fault domains on purpose. If you stay hosted, choose the region explicitly (Render Ohio vs Frankfurt, Herald EU, Vercel fra1) and test what breaks when you intentionally fail that region. If you own the fleet, implement the five-line CAPH shape above across three Hetzner locations with two server types, then kubectl drain one location and watch the workload reschedule. The drill is the document.

3. Price the next 15 hours before it happens. Put your hourly burn into the worksheet at the top and decide, with your co-founders, whether a $15k, $150k, or $1.5M 15-hour loss is an annoyance, an incident, or an existential event. The answer determines how much headroom, multi-region spend, and operational overhead is rational — which is the exact inverse of discovering the answer at 03:11 ET.

us-east-1 is not going away. As the oldest, largest AWS region it will keep attracting new services, new defaults, and new customers — and, every 12–24 months, a new incident report. The teams that sleep through the next one will not be the ones whose vendors finally achieved five nines. They will be the ones who drew a boundary small enough that a single region going dark is a problem for someone else's whole internet, not for theirs.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. A single Render-compatible API plus an open MCP server so humans and agents operate the same fleet. Star the repo on GitHub or provision your first three-node fleet across fsn1/nbg1/hel1 today.