Fly.io's July 9, 2026 Depot Builder Outage in IAD: What Renting a Shared Remote-Build Fleet Costs When the Region Itself Degrades

At 4:08 PM ET on July 9, 2026, fly deploy started hanging for anyone building in Fly.io's IAD region. Not failing outright — hanging, stuck on "waiting for depot builder…" with no clear signal whether to wait it out or bail. The fix landed at 16:18 UTC, roughly fourteen hours and ten minutes after the first report, and for part of that window a second problem — registry capacity constraints slowing image pushes and pulls — meant even the documented workaround was running on a degraded path.

This wasn't Fly's first Depot-builder incident of the year. It was the third in under five months. That recurrence, not just the outage itself, is the actual story: when your build step runs on a third party's regional fleet instead of your own machines, you don't just inherit that vendor's downtime — you inherit it on a schedule you don't control, with a workaround you have to already know about, and a bill that never shows up as a line item because "eat a slower build and a manual flag" doesn't have a dollar sign attached.

The Timeline: 14 Hours, 10 Minutes, One Flag​

Here's what Fly's own status page and the resulting community thread laid out for July 9:

The documented workaround, which spread through Fly's community forum under the thread title "Depot builder outage (and workaround!)," was a single flag: fly deploy --depot=false. That forces the CLI off Depot entirely and onto Fly's own legacy builder pool — the fallback existed, but a team only found out about it by searching a support forum mid-incident, not from an automated failover Fly's tooling triggered on their behalf.

What Actually Broke: The Architecture Behind "Waiting for Depot Builder"​

Fly's build step isn't Fly's anymore — it's Depot's. Since flyctl 0.2.102, fly deploy defaults to a Depot-hosted builder: a single-tenant VM that Depot provisions, runs BuildKit on, and tears down, with build cache persisted on Fly volumes so repeat builds reuse prior layers. Depot's own pitch for this is speed — "build your Docker images in seconds, not minutes" — and for a warm cache, that's real.

The part that matters for this incident is where that speed lives. Depot's cache is tied to a specific Availability Zone via NVMe-backed Ceph storage, specifically so the builder and its cache volume stay physically close and low-latency. Depot runs builders across 18 of Fly's regions, but each region's builder capacity is its own pool, provisioned and managed by Depot's control plane — not by Fly's own scheduler, and not portable across regions on demand. When IAD's pool degrades, every team whose builder and cache live in IAD is stuck, regardless of how healthy Fly's actual compute fleet is in that same region. The app-serving machines in IAD were fine. The build step routed to a co-located but separately-operated fleet was not.

That's the structural fact worth sitting with: "region-scoped" here doesn't mean "Fly's IAD data center had a bad day." It means a specific vendor's builder capacity in that region had a bad day, and there was no cross-region or cross-vendor path back to a working build except manually opting out of the vendor entirely.

Not a One-Off: Three Depot-Builder Incidents in Five Months​

If July 9 were an isolated blip, it would be a footnote. It isn't. Fly's own incident history shows a pattern specific to the Depot build path:

Three separate incidents, roughly one every six to seven weeks, all resolved the same way: fail over away from Depot, either by Fly flipping the default or by the customer passing a flag by hand. That's not noise — it's the build step's actual availability track record for 2026 so far, and it's a track record that lives entirely outside Fly's own infrastructure reliability numbers, because the builder fleet isn't Fly's infrastructure. It's rented.

What "Renting" Actually Costs​

Fly hasn't published an affected-tenant count, a failed-deploy count, or a service credit for any of these three incidents — which is itself worth noting: as a renter of someone else's build fleet, you don't get to see the bill for what broke, only that it broke. But the cost is still traceable in three concrete places:

The cache penalty on failover. Depot's whole value proposition is an incremental cache that turns a rebuild into a seconds-long operation. --depot=false doesn't fail over to an equivalent cache on a different vendor — it drops you onto Fly's legacy builder pool, which doesn't share Depot's cache. Every deploy forced onto that path during an incident pays close to a full rebuild, turning what should be a seconds-long deploy into one measured in minutes, for as long as the incident runs. Over 14 hours and 10 minutes, that's not one slow build — it's every deploy attempt from every IAD-routed team that tried to ship during that window.

The discovery tax. The fallback flag existed the whole time. Finding it required either already knowing Fly's CLI surface well enough to guess --depot=false, or searching the community forum mid-incident and landing on a thread a fellow customer had already written. That's real engineering time spent diagnosing someone else's outage instead of shipping — time that doesn't show up on Fly's status page, but shows up on every affected team's Friday.

The compounding blast radius. Because the registry capacity constraint hit in the same window, the fallback path itself was degraded for part of the incident — teams who successfully routed around the dead Depot builders could still hit slow or failed image pushes on the way out. A single-vendor build dependency and a single shared registry failing in the same 14-hour window isn't a coincidence a customer can plan around; it's two points of failure that happen to sit behind the same fly deploy command.

None of this required Fly's core compute or networking to fail. It required one third-party fleet, scoped to one region, to have a bad day — three times in five months.

The Structural Alternative: No Separate Vendor, No Separate Region​

The fix for this class of incident isn't "pick a better remote-build vendor." It's not routing the build step through a separate vendor's regional fleet at all.

On a self-hosted PaaS running on Cluster API-provisioned machines — Bex.co's model, for instance — the build for a git push deploy runs on the same owned node pool that's already running the rest of the platform. There's no Depot-equivalent control plane deciding builder capacity in a region you don't operate, and no separate incident to watch for on someone else's status page. If a node is up, it can build. If it's part of your fleet, you already know its capacity, its cache state, and its failure history, because you're the one operating it — not searching a support forum for the flag that routes around it.

That's a direct trade against Depot's genuine strength: Depot's incremental cache is fast in a way a self-hosted builder has to work to match. But "renting a shared remote-build fleet colocated in a region" and "running your build on the machines you already own" fail differently under load, and July 9 — plus February 20, plus June 25 — is what the first one costs when the vendor's regional capacity, not yours, is the thing that degrades.

What to Do Before the Next One​

For teams still on Fly, three things are worth doing now, not during the next incident:

1. Know --depot=false before you need it. It's the documented workaround, and it's faster to have it in your deploy runbook than to search for it mid-incident.
2. Don't assume cache locality survives a failover. If your build cache lives in one AZ via Depot, plan for a cold, slower build on any forced fallback — budget the time, don't discover it live.
3. Track the pattern, not just the incident. Three Depot-builder-specific incidents in five months is a frequency worth factoring into any capacity or reliability planning that assumes fly deploy is a Fly-only dependency. It isn't.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with the build step running on the same fleet as everything else. Star the repo on GitHub or deploy your first app today.