Fly.io Turned Off Its GPUs on August 1: Where GPU-Backed Tenants Go Now (With Real Numbers)

On August 1, 2026, fly deploy to a GPU Machine started failing for good. Fly.io's GPU fleet — A10s, L40S cards, and A100s that tenants used for inference, document processing, and agent sandboxes — went dark after a six-month sunset that began with a deprecation email in February. If you ran GPU workloads on Fly, you have already moved or you are running on borrowed time somewhere else.

This was not a surprise. It was the third step in a retrenchment two years in the making. And the invoice math of where those workloads went contains a finding most migration guides get wrong: the card itself was never the ripoff. The waste was everything around it.

The third shoe dropped on August 1​

Fly.io's pullback happened in three dated steps, each one narrowing what the platform is:

• October 7, 2024 — free allowances removed for new organizations, replaced by a one-time trial. The era of a free-tier Fly app ended.
• September 2025 — the Region Consolidation Project deprecated 17 regions, leaving 18. Tenants in places like Mumbai (bom), Boston (bos), and Rio (gig) woke up to "region is deprecated" errors and migrated to the surviving regions.
• February 2026 → August 1, 2026 — GPU deprecation emails went out on February 11; GPUs were fully deprecated as of July 31 and unavailable after August 1. The GPU docs page now 404s.

Each step stranded a different cohort, but the GPU shutoff stranded the most valuable one: tenants running inference and ML-adjacent workloads — exactly the workloads every platform wants in 2026. One tenant on the community forum described building a document processing pipeline (OCR, layout analysis) around L40S machines, chosen because Fly was "the easiest platform to build on: fly deploy, pick your region, done." Their analogy stuck with me: GPUs on a hosting platform are like a gym in a hotel — not every guest uses it, but the ones who need it will book a different hotel.

So where did they book? Before the destinations, it is worth understanding why Fly checked out — because their stated reasons contain the pricing insight.

Why Fly did it, in their own words​

Fly's CEO laid out the rationale in the 2025 post "We Were Wrong About GPUs," and it is unusually candid. Four points matter:

1. Dedicated GPU servers sat idle. GPUs terrified Fly's security team (a GPU is "just about the worst case hardware peripheral," with intense DMA outside the normal security boundary), so Fly ran GPU workloads on dedicated server hardware, unmixed with ordinary workloads. Those boxes were "drastically less utilized and thus less cost-effective than our ordinary servers."
2. They could never thin-slice a card. Fly's micro-VM stack (Intel Cloud Hypervisor with PCI passthrough) couldn't use NVIDIA MIG or virtualized GPUs — at one point engineers hex-edited closed-source drivers to trick them into thinking the hypervisor was QEMU. No thin-slicing meant no cheap tier: every tenant rented whole cards.
3. Developers want LLMs, not GPUs. Fly's thesis: most software developers reaching for AI just call an API. The system engineers who genuinely want raw CUDA are a niche — "GPU workloads for the 10,001st developer."
4. L40S inference was the only real user base. Fly even cut L40S prices in half because it was "the one part we have in our inventory people seem to get a lot of use out of." It still wasn't the bet paying off.

The key inference for anyone migrating: whole-card rental was the only shape Fly ever offered. You rented an L40S or you rented nothing — there was no half-card, no burst-to-zero, no bin-packing your sidecar inference next to someone else's. Every destination below should be judged on whether it fixes that shape problem, not just whether its sticker price is lower.

The cost table: what your workload costs in each new home​

Here is the concrete math for the typical stranded workload — an always-on inference service that used one L40S on Fly — plus the two workload shapes that change the answer. All figures are list prices; exchange rate roughly €1 ≈ $1.08.

Read the first row twice, because it is the finding most migration posts miss: for a single always-on L40S, Fly's price and a dedicated Hetzner box are at parity — roughly $900–910 a month either way. Fly wasn't overcharging for the card. The savings on owned hardware come from the two things Fly's architecture couldn't do:

• Right-size the card. Fly's cheapest GPU path was an A10 at ~$550 a month always-on. A 7B-to-32B quantized model that fits in 20 GB runs fine on an RTX 4000 at ~$200 a month dedicated. That is a 2.7x cut for the same tokens, available only where someone sells you a smaller card.
• Bin-pack the card. Fly gave one tenant one card. On your own box with time-slicing, MIG partitions, or DRA-based fractional scheduling, three inference services share one L40S at ~$300 each. The packing factor is the discount — 2 to 4x depending on how many workloads you can co-schedule.

And the sensitivity rule that decides between dedicated and serverless: a dedicated box wins when the card stays busy; per-second billing wins when it doesn't. At L40S-class serverless rates (~$1.00–1.50/hr), the break-even against an €838 fixed box lands around 80–100% utilization for a single workload.

Below ~500 GPU-hours a month, scale-to-zero serverless wins by a mile and you should not rent a card at all. Above that — or the moment you can keep one card busy with several workloads — owned hardware wins by the packing factor.

One more honest footnote: Fly's remaining CPU-only footprint is unchanged, so the "stay on Fly for the app, put GPUs elsewhere" split architecture is real and common. Price it as two bills, not one, and remember cross-provider data transfer for model weights is the line item everyone underestimates.

Where tenants actually went: five destinations​

From forum threads, migration commits, and provider lineups, stranded tenants sorted into five destinations. Each fits a different workload shape:

1. Modal — the smoothest serverless landing. Per-second billing, scale-to-zero, and the least ops burden of the dedicated-GPU options. Multiple migrants name it as the smoother experience versus RunPod. Best fit: bursty inference and batch jobs that once ran on Fly GPUs but never needed 24/7 cards. Watch: cold starts for large model weights, and per-second rates that punish always-on duty cycles.
2. RunPod — pods or serverless, your choice. Both persistent GPU pods and scale-to-zero serverless endpoints. The forum's document-pipeline tenant migrated here ("rough around the edges, gets the job done"), and at least one training pipeline pivoted to RunPod the week of the deprecation notice. Best fit: tenants who want to keep a pod-like mental model closest to Fly Machines. Watch: UX polish and region availability versus the hyperscalers.
3. Koyeb — the PaaS-shaped option. Git-push deploys with per-second GPU billing (A6000/L40S/A100 tiers), the closest thing to "Fly, but with GPUs still on." One project moved its hosted tier from Fly to Koyeb explicitly because of the GPU sunset. Best fit: teams that picked Fly for developer experience and want it back. Watch: volume size caps and a smaller region map than Fly at its peak.
4. GCP Cloud Run GPUs and other hyperscaler serverless GPUs. The enterprise-answer: GPU-backed scale-to-zero behind an existing cloud bill and compliance story. Best fit: teams already on GCP or needing SOC 2 / HIPAA paperwork with their inference. Watch: premium per-second rates and the egress bill for weights.
5. Owned hardware (Hetzner dedicated) under your own scheduler. GEX44/GEX130-class boxes, EU-only, monthly rental, one GPU per server. Best fit: always-on inference at high utilization, especially multiple services you can bin-pack per the table above. Watch: everything in the next section — you are the GPU platform team now.

Notice the pattern: nobody moved to a second generalist PaaS with GPUs as a side dish. Every destination is either GPU-native (Modal, RunPod) or infrastructure you control (Hetzner, GCP). The hotel-gym era — GPUs as a checkbox feature on an app platform — ended with Fly's fleet.

What the owned-hardware move honestly requires​

The cost table makes owned hardware look like free money at high utilization. It is not free; it is a platform-engineering project. Here is the checklist of what Fly used to do for you that you now own:

• GPU scheduling. Kubernetes with the NVIDIA device plugin at minimum; Dynamic Resource Allocation (DRA) with time-slicing or MIG partitions if you want the bin-packing numbers from the table. Fractional-GPU scheduling is the entire economic engine of the move — skip it and you just rebuilt Fly's cost structure with extra steps.
• An inference server. vLLM or TensorRT-LLM for LLMs, TGI or Triton for the general case — with continuous batching, KV-cache tuning, and quantization (AWQ/GPTQ) doing more for your $/token than any provider switch.
• Model-weight storage and loading. Fly's "app servers, GPUs, and object storage under the same top-of-rack switch" was a genuine advantage. On Hetzner you need fast local NVMe plus an object store (Hetzner Object Storage or self-hosted) and a warm-cache strategy, or every deploy re-downloads tens of gigabytes.
• Autoscaling and rollouts. No more fly deploy with built-in blue-green. You need image builds, a registry, rollout strategy, and GPU-aware autoscaling (KEDA + cluster-autoscaler on your node pool, or scale-to-zero with Knative/KServe if you want the serverless shape back).
• Edge, TLS, and regions. Hetzner GPU boxes live in Germany and Finland. If your users are elsewhere, budget for latency, a CDN, and anycast or regional routing you now design yourself. Fly's 18 surviving regions are still a real product advantage for the CPU side — which is why the split architecture persists.
• Observability and drivers. NVIDIA driver versions, CUDA compat, DCGM metrics, VRAM-oom alerts. Pin driver + CUDA + framework versions in your node image; "latest" is how you meet a Sunday-morning incompatibility.

None of this is exotic — it is standard platform engineering. But budget two to four weeks of it before the first production token, and do not compare the €838 sticker against Fly's $910 without pricing your own time to build the row above.

The pattern: don't get evicted twice​

Step back and the three-step timeline reads as one story: free tier (2024), regions (2025), GPUs (2026). Each cut removed the thing that was furthest from Fly's core business of fast-boot CPU Machines on owned hardware. GPUs were never the core business — they were the hotel gym, loss-making at Fly's utilization, unfixable inside Fly's hypervisor constraints, and serving a niche Fly's own credo ("design for 10,000 developers, not 5–6") told them to stop serving.

That is not a criticism. It is the predictable lifecycle of any feature that is a side dish on someone else's platform. The tenants who got hurt worst were the ones who built load-bearing infrastructure on a side dish — a document pipeline with no second GPU supplier, a training loop with Fly as the only compute assumption.

The decision rule going forward is simple: run load-bearing GPU workloads only where GPUs are the core business or the hardware is yours. Modal and RunPod live or die on GPU margins; their incentives point at keeping your card lit. Your own Hetzner box cannot be deprecated by email. A generalist PaaS can always do this math again — and the next one will, the moment its own utilization numbers say to.

If you are sketching the owned-hardware column of the table above and want the deploy experience without rebuilding the platform team, that gap is exactly what a self-hosted PaaS on machines you own is for: git-push deploys, declarative machine lifecycle, and GPU scheduling you control, on hardware nobody can email you about.

What to do this week​

If you still have Fly GPU references in your infrastructure — fly.toml GPU sections, region pins to GPU-only regions, automation that assumed fly deploy provisions accelerators — the cleanup order is: first, pick your destination from the workload shape (bursty → serverless, always-on single → parity math, always-on plural → owned + bin-packing); second, move weights and images before touching compute; third, delete the GPU assumptions from your deploy pipeline so the next deprecation email finds nothing to break.

August 1 already happened. The tenants who treated the February email as a migration deadline are fine. Everyone else just paid six months of sunset pricing to learn that the cheapest GPU is the one you don't rent twice — right-sized, bin-packed, and on hardware that answers to you.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.