Dora NodaOn Tuesday morning, September 15, every fly deploy routed through Fly.io's Sydney and Johannesburg regions started failing. The cause wasn't your Dockerfile, your region, or your code — it was Depot, the third-party build fleet that has been Fly's default remote builder for the past year. For about an hour and a half, teams in those regions couldn't ship unless they knew about a single CLI flag. And here's the number that matters: past roughly 1,700 build-minutes a month, renting those build minutes at Depot's list rates costs more than owning the box they run on.
That is the bill for a rented build fleet, twice over. There is the invoice — four cents a minute, metered — and there is the outage ledger, which now shows eight Depot-related deploy disruptions in twelve months. This post prices both, from the incident records, and sketches what a fleet-local BuildKit setup on hardware you own changes about the math.
An 85-minute Tuesday: what happened on September 15
Pingoru's incident record for Fly.io on September 15, 2026 tells a compact story. At 04:08 UTC the tracker picked up "Depot builder failures" against Fly's Deployments component; Fly's own status page identified the issue at 04:45 UTC as broken deploys via Depot for users connecting through its SYD and JNB regions. The published workaround was to bypass Depot entirely:
fly deploy --depot=false
# or, on the experimental path:
fly deploy --buildkitA fix was in and being monitored by 04:51 UTC, and the incident was marked resolved at 05:34 UTC — a window of about an hour and 25 minutes during which the default fly deploy path was broken for two regions.
Two things are worth noticing. First, the failure was regional, not global: if your traffic entered through North America or Europe, Tuesday was uneventful. Shared remote-build fleets fail in blast radii, and this time the radius was SYD plus JNB. Second, the workaround existed but had to be known: --depot=false reroutes your build to Fly's older native remote builders, the "rchab" system Fly built back in 2021. Teams that had been through a previous Depot incident typed it from memory. Teams that hadn't waited.
Before the history, the cost headline, since it's the reason to keep reading:
| Monthly build volume | Rented (Depot list rates) | Owned (Hetzner CCX33 BuildKit box, ~€63/mo) |
|---|---|---|
| Light — 500 min | $20 (Developer plan, fits) | ~€63 — renting wins ~3x |
| Typical — 3,000 min | $200 (Startup plan, fits) | ~€63 — owning wins ~3x |
| Heavy — 10,000 min | ~$400 ($200 + 5,000 × $0.04) | ~€63 — owning wins ~6x |
Break-even sits near 1,700 build-minutes a month. The full methodology — and the honest caveats about utilization — is below. But the shape of the answer is that simple: light teams should rent, everyone past a couple of daily deploys should price the box.
Two months earlier in IAD — and six more times in the log
The September outage was the second Depot builder failure in about two months. The first was July 9, 2026, and Fly's own infra log entry for that day — "A couple of deployment issues," posted 15:11 UTC — is unusually candid about what went wrong.
In IAD, flyctl deploy would wait seemingly indefinitely for a Depot builder to become available. Fly first blamed a single host under I/O pressure, then discovered the problem was fleet-wide in the region: Depot builder machines are large, and under capacity constraints they can't start against their existing volumes because the host has no spare capacity — while the auto-migrate-on-start behavior that saves ordinary machines is explicitly disabled for machines and volumes as large as builders. The fix, before new capacity arrived, was to purge existing Depot builders from IAD so they'd be recreated in other North American regions.
Meanwhile Fly's Docker registry started throwing 5xx errors at the same time — saturated TAP interfaces above 1 Gbps on the registry machines — compounding a bad deploy day. And the status page went up late, because the team initially assumed the builder failures were part of an already-tracked downed-host issue. StatusGator clocks the IAD builder delays at 55 minutes.
Note the root cause carefully: July was Fly-side capacity planning, not Depot's code. That's precisely the point. When you rent the build fleet, the failure domain is the whole stack between your fly deploy and a working builder — Depot's database, Fly's capacity headroom, Fly's orchestrator API, the registry — and you control none of it. The July postmortem's own follow-ups (better Depot placement monitoring, geo-region aliases for builder placement so local capacity crunches stop becoming regional deploy failures) are things the vendors must build; tenants just wait.
And "second in two months" understates the pattern. Fly's infra log records every Depot-related deploy disruption going back to the partnership's default rollout. The ledger:
| Date | Duration | Cause |
|---|---|---|
| Sep 18, 2025 | Morning (minutes unrecorded) | Depot default ~2 weeks old; Fly orchestrator API incompatibility broke Depot builds; feature-flag cutover to native builders |
| Oct 22, 2025 | ~5 min | Depot code change broke auth-token handling |
| Jan 23, 2026 | ~1 hour | Depot database outage; --depot=false required |
| Feb 10, 2026 | ~45 min | Depot database issue; status-page workaround to native builders |
| Mar 26, 2026 | ~90 min window | FRA capacity crunch; most visible to Depot builder users |
| May 8, 2026 | "A couple hours" | Depot outage; platform-wide fallback to native builders |
| Jul 9, 2026 | ~55 min (plus registry 5xx) | IAD capacity; builders couldn't start, no auto-migrate |
| Sep 15, 2026 | ~85 min | SYD/JNB Depot builder failures |
Eight incidents in twelve months, six of them in 2026 alone, totaling roughly seven-plus hours when the default deploy path was degraded or dead. Every one of them had the same tenant-side experience: deploys hang or fail, and your only lever is a flag that opts you out of the default backend — a backend Fly has said will eventually be the only backend, with the standard remote builder likely retired once the Depot migration is fully verified. The escape hatch has a sunset horizon.
What actually breaks when the rented fleet fails
Strip the eight incidents to their mechanics and four structural facts emerge:
1. The per-region fleet is a single point of failure. Depot runs your builds inside its own Fly.io organization, on shared builder machines per region. When SYD's builders fail, every SYD deploy fails identically — there is no "my builder" to restart, no alternate pool to pick. Contrast a fleet-local setup, where N build nodes means N independent failure domains you can drain one at a time.
2. Tenants can't reroute; they can only opt out. The --depot=false flag doesn't move your build to healthy Depot capacity in another region — it switches backends entirely, to an older system with slower builds and no long-term future. Real rerouting (geo-region aliases for builder placement) is on the vendors' roadmap, not in your hands.
3. Builder-sized machines get second-class scheduling. The July postmortem's most telling detail: machines and volumes as large as builders are excluded from auto-migrate on capacity errors. The exact machines your deploys depend on are the ones the scheduler is least able to rescue. On your own fleet, you'd pin builders to reserved nodes and this class of failure wouldn't exist.
4. The registry compounds everything. July's simultaneous registry 5xx errors are the reminder that "the build fleet" is really a chain — builder, registry, orchestrator API — and rented chains fail in correlated ways during capacity crunches. Your deploy needs all three; you operate zero.
None of this is an argument that Depot or Fly are badly run — Fly's infra log is one of the most transparent in the industry, and every incident above got a workaround within minutes to hours. It's an argument about agency: when the fleet is rented, your incident response is a status page and a flag. When it's yours, it's a runbook.
The dollar math, in full
Depot's public pricing (verified September 2026): the Developer plan is $20/month including 500 Docker build minutes; Startup is $200/month including 5,000; additional Docker build minutes are $0.04 each; cache storage is $0.20/GB/month. Fly users don't pay Depot directly — Fly bundles builds, with a planned 300 free build-minutes a month per the original announcement — but Depot's list rates are the market price of a rented build minute, and they're what you pay the day you outgrow any bundle or buy direct.
The owned alternative: a Hetzner CCX33 — 8 dedicated vCPUs, 32 GB RAM, 240 GB NVMe — at about €63/month post-2026 pricing. One such box running BuildKit 24/7 offers 43,200 available minutes a month. Walk the three volumes:
- Light (500 min/mo — a side project, a few deploys a week). Rented: $20, inside the Developer plan. Owned: ~€63 for a box that's idle 99% of the time. Renting wins by ~3x, and the box's idle waste is real money. This is the one segment where the outage ledger is the only argument for owning.
- Typical (3,000 min/mo — a team deploying ~10× a day at ~10 min a build, or 20× at ~5 min). Rented: $200, inside the Startup plan. Owned: ~€63. Owning wins by ~3x — roughly $1,600/year back — before counting cache storage overages on the rented side.
- Heavy (10,000 min/mo — monorepo, matrix builds, or a busy agency box). Rented: $200 + 5,000 × $0.04 = ~$400/month. Owned:
€63, and even a second box for redundancy (€126 total) wins by ~3x. Owning wins by ~6x on one box.
The sensitivity that matters is utilization. Builds are bursty: a team doing 3,000 minutes on a 43,200-minute box uses 7% of it. At that utilization the box's effective rate is ~$0.02/minute — still half of Depot's $0.04, with warm caches and zero queueing. The crossover where renting wins outright is below ~1,700 minutes a month (where $68 ≈ €63 buys the same minutes at $0.04). Above it, you're paying a premium for elasticity you use in spikes and availability you don't control.
Two honest caveats. First, the box needs operating: BuildKit cache GC, disk-full alerts, multi-arch (QEMU or native ARM runners), and base-image pulls all become your runbook — budget a day to set up and an hour a month to keep. Second, Fly's bundle means this math is currently latent for Fly users, not billed. Price it anyway: bundles reprice, free allowances shrink, and the outage ledger above is denominated in blocked deploys, not dollars. A team shipping 10× a day lived through six "will my deploy go through" windows in 2026 so far. No per-minute rate captures that; multiply your deploy frequency by your cost of a blocked release and add it to the rented column.
What fleet-local BuildKit actually buys
The owned alternative isn't "build on your laptop" — it's the same remote-builder architecture, on machines you control. The pattern is well-trodden: run BuildKit as a daemon on your build nodes, expose it over mTLS, and point buildx at it with the remote driver:
docker buildx create --name fleet \
--driver remote tcp://build-01.internal:1234
docker buildx use fleetRegistry-backed BuildKit cache (--cache-to/--cache-from against your own registry) replaces Depot's distributed cache; every build node warms from the same layer store, and cache locality is a LAN round-trip instead of a cross-region one. Builders live in the same region — the same rack, if you like — as the fleet they serve, so a SYD-style regional degradation in someone else's build cloud is simply not in your failure domain. And multi-node means maintenance without downtime: drain one builder, upgrade it, bring it back, while deploys flow through the other.
What it doesn't buy is freedom from operations. You own cache-GC tuning, disk-capacity planning, builder upgrades, and the ARM story (either emulation-friendly pipelines or native ARM nodes alongside x86). For a platform team that's a weekend project with a thin runbook; for a solo dev shipping twice a week, it's overkill — which is exactly what the 1,700-minute crossover says.
The decision checklist, then, is short: if you burn past ~1,700 build-minutes a month, if a blocked deploy window costs you more than an hour of toil, or if you've typed --depot=false from memory more than once this year, the rented fleet is no longer saving you money or time. It's just where your deploys go to wait.
Eight Depot incidents in a year is the pattern that makes teams price the box. Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with the build fleet under the same roof as the workloads. Star the repo on GitHub or deploy your first app today.
