The Self-Hosted GPU Breakeven Point: Why ~50M Tokens/Day Is Where Owning H100s Beats Renting Inference APIs

Rent a dedicated H100 for a month and the bill doesn't care whether you sent it one token or one billion: it's a fixed $2,000-$2,500, every month, whether the card sits at 90% utilization or 9%. A hosted inference API is the opposite — every token costs the same whether you send one or a billion. Cross those two lines and there's exactly one point where the fixed cost of an idle-tolerant GPU beats the variable cost of a metered API.

Run the numbers on a single dedicated H100 serving a 70B-class open-weight model, and that crossover lands at roughly 50 million tokens per day — but only against a typical mid-tier hosted API. Against the cheapest budget open-weight endpoints, a single H100 never wins. Against frontier-model pricing, it wins at a fraction of that volume. The number in the headline is real, but it's one point on a curve, and the curve is the actual story — especially for a platform deciding whether to give every tenant's AI agent its own GPU sandbox or proxy everything to someone else's API.

The Math: Building the Breakeven Model​

Two costs need to be on the table before "breakeven" means anything.

The self-hosted side is a fixed cost. Dedicated H100 SXM capacity from a GPU cloud (Lambda Labs, RunPod, CoreWeave, or a neocloud reseller) has stabilized in 2026 in the $2.85–$3.50/hour range for reserved capacity, with outliers like Hyperbolic advertising lows near $1.49/hour and hyperscaler on-demand pricing running $4–$11/hour. Take $3.00/hour as a representative dedicated rate: run it 24/7 and that's $2,190/month, or about $72/day — a number that doesn't move whether the GPU serves one request or ten thousand. (Buying the card outright tells the same story from a different angle: an H100 PCIe runs $25,000–$31,000 and an SXM5 unit $35,000–$40,000+, and once you amortize that over three years and add the power, cooling, and facilities overhead that owning bare metal requires — commonly cited at 2.5–3x the raw hardware cost — a purchased card converges on roughly the same $2,500–$3,500/month all-in figure as renting dedicated capacity. Ownership only pulls ahead of renting past 10,000+ GPU-hours/month sustained for multiple years, which is a data-center operator's math, not a single team's.)

The hosted-API side is a variable cost. Every provider charges per million tokens, so the question becomes: how many tokens can that $72/day fixed-cost H100 actually serve, and at what price does the API-per-token math cross the GPU's per-token math?

Throughput is the piece most breakeven posts wave away, so it's worth being explicit about the estimate: a published vLLM benchmark serving Llama 2 70B on 4x A100 GPUs sustained roughly 2,200 tokens/second with 256 concurrent users — about 550 tokens/second per A100. H100s deliver roughly 2x an A100's throughput on the same model class once FP8 and TensorRT-LLM kernels are in play, which puts a single H100 at a conservative ~1,100 tokens/second sustained under realistic concurrent load (not a synthetic peak benchmark). That's ~95 million tokens/day of raw capacity at 100% utilization — the ceiling this whole model is built against.

Crucially, both sides of this comparison need to be the same class of model. A self-hosted 70B open-weight model isn't a substitute for a frontier reasoning model — it's a substitute for whatever hosted endpoint serves a comparable 70B-class model. Comparing a self-hosted 70B card against GPT-5-class API pricing isn't a fair fight; comparing it against a hosted endpoint for a similarly-sized open-weight model is. The table below keeps that distinction explicit by tier.

Where the Breakeven Actually Lands, Tier by Tier​

Divide the fixed $72/day by the number of tokens actually served that day, and you get the self-hosted effective price per million tokens. Set that equal to a hosted API's price and solve for the daily volume where the lines cross:

That middle row is where the "~50M tokens/day" headline comes from, and it's real — but only against a representative mid-market hosted rate. It is not a universal number, and the honest version of this post has to say so plainly: against the cheapest budget open-weight APIs, one self-hosted H100 never wins on pure token economics, full stop. Even running that card at 100% utilization around the clock, its effective cost floor is about $0.76/M tokens — five times the price of a $0.15/M budget endpoint. Beating budget pricing requires either cheaper-than-dedicated GPU sourcing, or — as the next two sections cover — pooling enough concurrent demand that a single card's fixed cost gets spread across enough tokens to undercut even the cheapest metered rate.

Why Idle Capacity Is the Real Risk, Not GPU Price​

The breakeven table above assumes you actually hit the utilization it requires. Miss it, and the fixed-cost bet inverts fast, because the $72/day doesn't shrink when traffic does:

A GPU running at 10% load isn't a discount — it's the most expensive tier in the whole table. This is exactly why NVIDIA's GPU Operator ships DCGM (Data Center GPU Manager) as a first-class Kubernetes component by default: DCGM_FI_DEV_GPU_UTIL tracks the fraction of time the GPU isn't idle, and DCGM_FI_PROF_SM_ACTIVE shows what fraction of streaming multiprocessors are actually doing work during a request, exposing headroom that raw utilization numbers can hide. The financial stakes of watching these numbers are not abstract — one widely cited cluster-operations case found that even a 10-percentage-point utilization improvement across a 100-GPU fleet at $2/hour saved roughly $175,000 a year, purely by closing the gap between paid-for and used capacity. Below the breakeven line, idle H100 capacity — not sticker price — is the dominant cost risk of self-hosting.

The PaaS Angle: Bin-Packing Tenants to Cross the Line​

Here's the problem with all of the math above applied to a single team: almost no single team's inference traffic is smooth enough to sustain 50%+ utilization around the clock. Real workloads are bursty — spiky during business hours, quiet overnight, unpredictable around product launches — which is exactly the traffic pattern that drags a dedicated GPU's effective cost toward that $7.58/M worst case.

A multi-tenant platform has a structural answer that a single team doesn't: pool many tenants' AI-agent workloads onto shared GPU capacity, and the aggregate traffic smooths out in a way individual tenants' traffic never does. Kubernetes made this concrete in 2026 — Dynamic Resource Allocation (DRA) reached general availability in Kubernetes 1.34, replacing the old device-plugin model (where a GPU was an opaque nvidia.com/gpu: 1 resource) with structured ResourceClaims that expose memory, compute capability, and topology, letting the scheduler bin-pack workloads onto GPUs the way it's long bin-packed CPU and memory requests. NVIDIA has since donated the DRA driver for GPUs to the CNCF, putting it under community governance rather than single-vendor control. The scheduling layer to make cross-tenant bin-packing a platform primitive, not a bespoke project, now exists.

This is the load-bearing insight for a self-hosted PaaS weighing "give every tenant its own GPU" against "proxy every tenant to a hosted API": neither extreme is right. A single tenant's traffic essentially never clears the utilization needed to make a dedicated card pencil out — but a platform that bin-packs many tenants' AI-agent sandboxes onto shared, owned GPU-node capacity can realistically sustain 70-90% utilization across the pool, the range where self-hosting stops being a bet and starts being the obviously cheaper option against everything but the rock-bottom budget tier. The GPU-node capacity itself typically comes from dedicated neocloud instances (Lambda, RunPod, CoreWeave, Hyperbolic) or owned bare metal rather than Hetzner specifically — Hetzner doesn't currently sell H100 SKUs — but the underlying economics (fixed-cost commodity hardware, amortized across pooled demand) are the same commodity-ownership logic Hetzner made popular for CPU nodes, now applied to GPUs.

What This Means for a Self-Hosted Platform​

The decision framework falls out of the table above, and it's simpler than the model that produced it:

• Below ~15-20M tokens/day of aggregate, poolable demand: proxy to a hosted API. No pool of tenants realistically gets a shared GPU fleet's utilization high enough to beat even premium-tier API pricing at that volume, and the operational overhead of running inference infrastructure isn't worth it yet.
• Above ~50M tokens/day of aggregate demand, with bin-packing that can sustain 50%+ utilization: owning or dedicated-renting GPU-node capacity starts winning against mid-tier hosted pricing, and the margin widens fast as pooled utilization climbs toward 80-90%.
• Against the cheapest budget open-weight tier specifically: don't expect to win on a handful of GPUs. That requires either scale most platforms won't reach for a while, or accepting that budget-tier proxying stays the right call for that slice of traffic indefinitely.

What a platform needs operationally to act on any of this is the same DCGM-fed utilization signal from section three, surfaced per-tenant rather than per-cluster — the dashboard that tells an individual team whether their workload is contributing to a profitable shared pool or still cheaper served from someone else's API, instead of leaving that as a guess.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. If you're weighing owned GPU-node capacity against hosted inference APIs for an AI-agent platform, the same self-hosted-PaaS economics apply to the rest of your stack. Star the repo on GitHub or deploy your first app today.

Sources​

• Inference Unit Economics: The True Cost Per Million Tokens
• Self-Hosted LLM vs API: The $4,200/mo Break-Even Point
• Self-Hosting Open-Weight LLMs: 2026 Decision Guide
• H100 Rental Prices Compared: $1.49-$6.98/hr Across 15+ Cloud Providers (2026)
• NVIDIA AI GPU Prices: H100 ($27K-$40K) & H200 ($315K/8-GPU) Cost Guide
• 2026 Cost of Renting or Buying NVIDIA H100 GPUs for Data Centers
• H100 GPU Cost In 2026: Buy, Rent, And Cloud Pricing Compared
• Turbocharging Llama 2 70B with NVIDIA H100
• Making GPU Clusters More Efficient with NVIDIA Data Center Monitoring Tools
• Kubernetes Primer: Dynamic Resource Allocation (DRA) for GPU Workloads
• Kubernetes AI Infrastructure in 2026: GPU Scheduling & Production Realities
• NVIDIA NIM API Pricing 2026: Free Tier, 40 RPM & Real Cost