Hetzner's €889 RTX PRO 6000: What Self-Hosted AI Inference Actually Costs When You Already Own the Fleet

If you already run your apps on a Hetzner fleet via Cluster API, adding a GPU node for inference looks deceptively simple: one more machine type in the same provisioning loop, same network, same control plane. Hetzner's top GPU server in 2026, the GEX131 with an NVIDIA RTX PRO 6000 Blackwell Max-Q and 96GB of GDDR7, costs €889 per month — or €1.42 per hour if you rent by the hour — with no setup fee. The question is not whether you can add it. It is whether the flat monthly meter beats paying a hosted inference API per token once you do.

Spoiler: at sustained load it wins by an order of magnitude. At idle, it loses embarrassingly. The breakeven lands far lower than most teams guess.

Here is the math, with no hand-waving.

The Answer Up Front​

A single RTX PRO 6000 can serve roughly 100–300 tokens per second on mid-size open models in FP8, depending on model size and batching. Run it 24/7 and you produce 260M to 780M tokens a month on one card. At €889 flat, that is:

Compare that to hosted inference in mid-2026:

• Budget open-weight APIs (DeepSeek, Together, Featherless): $0.14–$0.80/M input tokens. The cheapest way to call someone else's model.
• Mid-range hosted (GPT-4.1 mini/nano, Gemini Flash): $0.10–$0.75/M input, $0.40–$4.50/M output.
• Flagship APIs (GPT-5 / GPT-4o at $2.50/M input, Claude Sonnet 4.5 / Opus 4.6 at $3–$5/M input and $15–$25/M output): $2.50–$5.00/M blended at the input side, far more once output tokens dominate.

The implication is immediate. If you already own the fleet, a GEX131 crushes flagship API pricing once you sustain about 15–25M tokens a month — roughly a few hours of real user traffic a day. Against the cheapest open-weight APIs at $0.14–$0.28/M, you need substantially more volume — about 120–180M tokens a month — to justify the flat commit. And below ~10M tokens a month, even the cheapest self-hosted card is pure waste: you are heating a room to serve a trickle.

That is the whole decision in one table. The rest of this post shows how we got there, and where the naive read breaks.

What Hetzner Actually Sells​

Hetzner's GPU line in 2026 is short and easy to reason about:

• GEX131 — the flagship. NVIDIA RTX PRO 6000 Blackwell Max-Q, 96GB GDDR7, Intel Xeon Gold 5412U (24c/48t), 256GB DDR5 ECC (expandable to 768GB), 2×960GB Gen4 NVMe Datacenter Edition, 1 Gbit/s networking. €889/month or $989/month, or €1.4247/hour billed hourly. No setup fee since December 2025. Hetzner explicitly pitches the hourly option for time-limited projects — a few hours or days without a monthly commitment.
• GEX44 — the entry tier. NVIDIA RTX 4000 SFF Ada Generation, roughly €184/month. The card is far smaller (20GB), but the contract shape is identical: flat monthly, no per-second meter, no stop-to-save button while the machine is provisioned.
• Between them sit the GEX130 (RTX 6000 Ada, 48GB) and older GEX lines. For inference cost modeling the two extremes — €184 and €889 — bound the real decision.

The billing model is the point. Hetzner does not meter GPU seconds. You pay the monthly (or hourly) rate whether the card is at 100% or 0%. On hourly, you can delete the server and stop paying — but you cannot pause it and resume with state intact. There is no "scale to zero and pay nothing until the next request" the way Fly.io Sprites or Modal offer for sandboxes, or the way a per-token API charges nothing when no one is chatting.

That single fact — flat cost, no idle refund — is what makes burst analysis (later) the hinge.

For a team already running a Cluster-API-provisioned Hetzner fleet (CAPH), the GEX131 is not a new vendor or a new VPC. It is another machine deployment in the same cluster: a GPU node pool beside your existing CPU pools, same private network, same Kubernetes with a nodeSelector or tolerations for the GPU workload. The marginal operational tax is an extra machine class, not a second cloud.

How Many Tokens a Single 96GB Card Can Actually Serve​

The RTX PRO 6000 Blackwell is not a toy. At 96GB it holds Llama-70B at INT8, Qwen2.5-32B at full precision, or a pair of smaller models side by side. It is also why Hetzner prices it where it does — the card alone lists at $13,250 at NVIDIA's own store in mid-2026.

Third-party vLLM benchmarks on a single RTX PRO 6000 (FP8, production-ready settings) tell a consistent story:

• Large dense model (Qwen3.5-27B, Devstral 24B): ~100–118 tok/s sustained, ~69 tok/s per single sequential request at 1K context, TTFT under 120ms.
• Mid-size (Ministral 14B): ~300 tok/s at concurrency 4 with 1K context.
• Small (Ministral 3B): ~1,030 tok/s at concurrency 6.
• MoE 120B-class (Nemotron 3 Super 120B-A12B in NVFP4): ~178 tok/s with 96GB, and scaling is sublinear with context length — long contexts (256K) pull per-request generation down to 20–45 tok/s.

The independent "one RTX PRO 6000 matches four RTX 5090s" benchmark is real: a single card did 118.7 tok/s where four RTX 5090s pooled to 120.5 tok/s on a 230B MoE model, at a quarter of the power.

For cost modeling, pick a conservative baseline of 100 tok/s blended across request shapes. That is honest for a mixed workload serving a 14–30B model with some long contexts and moderate concurrency. Heavily optimized small-model serving (3–7B) does 3–10× better; long-context heavy workloads do worse. We will use 100 tok/s as the reference and note where a different model size moves the needle.

At 100 tok/s, the monthly token budget is:

• Per second: 100 tokens
• Per hour: 360,000 tokens
• Per day (24h): 8.64M tokens
• Per 30-day month, sustained: ~259M tokens

A 300 tok/s mid-size setup triples that to ~777M tokens/month. The table in the introduction used the 100 tok/s number, so it understates what a well-batched small-model deployment can do and overstates what a 70B-at-full-context deployment will actually achieve.

What Hosted APIs Actually Charge in 2026​

API pricing in 2026 spans roughly a 600× range. The number you compare against matters more than the self-hosted number itself:

Flagship (what most teams default to):

• GPT-5 / GPT-4o family: $1.25–$2.50/M input, $10–$15/M output.
• Claude Sonnet 4.5/4.6: $3.00/M input, $15/M output. Opus variants: $5/M input, $25/M output.
• Blended cost for a typical chat (1K input + 500 output) lands around $6–$10/M tokens when you count both sides.

Mid-range / budget hosted:

• GPT-4.1 nano: $0.10/M input, $0.40/M output. GPT-4.1 mini: $0.40/M / $1.60/M.
• Gemini Flash-class: similar bottom tier.
• DeepSeek V4 Flash via open-weight API providers: $0.14/M input, $0.28/M output — the floor of managed pricing.
• Together / Featherless open-model APIs: $0.27/M input, $0.80/M output.

The modifiers that change the real bill:

• Prompt caching (Anthropic, OpenAI): up to 90% off cached prefix — can quarter the effective rate for repeated system prompts.
• Batch API (50% off): irrelevant for realtime chat, huge for offline evals.
• Output tokens cost 3–5× input tokens — self-hosted cards do not make that distinction; an output token and an input token cost the same iron.

A self-hosted GEX131 at €4–€8/M (at 40–80% utilization) sits below every flagship API and roughly on par with the cheapest open-weight managed APIs only once you push enough volume to keep the card busy. That is the tradeoff in one sentence.

The Breakeven Math: Three Real Workloads​

Assume a team already runs a Hetzner fleet for its web/worker services. The GPU node is additive — you are not also paying for a separate Kubernetes control plane or network. Take the GEX131 at €889 (~$989) flat.

Two observations survive every sensitivity pass:

1. Against flagship APIs, the breakeven is low. Even at 30M tokens/month — a modest AI feature, not the whole product — self-hosting saves ~$130/month after the hardware, and the gap widens linearly. At 100M tokens/month you are saving thousands.

2. Against the cheapest open-weight managed APIs, the breakeven is high. You need to sustain roughly 150–200M tokens/month before a single GEX131 undercuts a $0.14/M provider even on blended cost. If your model choice is flexible and you are happy with DeepSeek-class pricing through a managed proxy, self-hosting is not a cost win until volume is serious.

Model size shifts this. Serve a 3B/7B model at 500–1,000 tok/s and the GEX131's monthly capacity climbs past 1B tokens — then it undercuts even the cheapest API at half the utilization. Serve a 70B model at full context with 30–50 tok/s effective and the card's monthly ceiling drops to ~80–130M tokens — then you may need a second card sooner.

Where Bursty Load Flips the Answer​

The tables above assume steady load spread evenly across the month. Real inference traffic is not steady. It spikes at 9am when users log in, craters at 2am, dips on weekends, and may sit idle between deploys.

A flat-monthly GPU charges you for the crater. A per-token API charges you for nothing when idle. That asymmetry is the whole "no stop button" problem the TODO item names.

Effective cost vs utilization (GEX131, 100 tok/s reference, €889):

• 100% utilization: €3.43/M
• 70%: €4.90/M
• 40%: €8.58/M
• 20%: €17.15/M
• 10%: €34.33/M
• 5%: €68.66/M

At 10% average utilization — a very normal number for a bursty internal tool that is hot for four hours a day and cold the rest — the GEX131's effective rate is already worse than GPT-4o's flagship pricing and 100× worse than the cheapest API. You are renting a concert hall for a meeting that happens twice a week.

The hourly option (€1.4247/hr) helps but does not solve the burst shape cleanly. An always-on hourly card costs ~€1,026/month — actually more than the monthly contract. The win is tear-down: run the GPU only during business hours (say 10h/day × 22 workdays = 220h) and you pay ~€313 for those hours. That cuts the idle tax roughly in third — but now you need automation to provision and deprovision the machine (Cluster API can do this), warm model weights on start (minutes, not seconds), and tolerance for cold-start latency. Hetzner's own docs frame hourly as "a few hours or days for time-limited purposes," not as a minute-by-minute autoscaler.

Compare that to hyperscaler on-demand GPUs: an AWS g5.xlarge (A10G 24GB) is ~$1.01/hr and a p5 H100 slice is ~$6.88/hr, both stop-when-stopped. Hetzner's hourly is cheaper per-GPU-hour than AWS, but AWS lets you autoscale to zero with managed services. The tradeoff is not hourly rate — it is what happens at zero. On Hetzner, zero means you deleted the server. On a per-token API, zero means you paid zero and kept zero infrastructure warm.

The practical implication for bursty workloads:

• Steady & predictable (>40% average utilization): self-hosted flat monthly dominates.
• Diurnal burst (office-hours-heavy with nightly idle): self-hosted still wins if daytime volume is high enough, but consider scheduled hourly teardown or hybrid — keep a small GPU warm for baseline and burst overflow to a per-token API.
• Sparse / event-driven (<15% average): hosted API wins outright. Do not provision a flat card for a cron job that fires a few times a day.

This is also where the entry GEX44 at ~€184 matters. For a bursty low-volume workload that wants data ownership without the €889 commitment, the cheaper card moves the breakeven down by ~4.8×. Its effective cost at 10% utilization is ~€7.10/M rather than €34/M — suddenly competitive again. The right sizing question is often "which card," not "card or API."

The Ops Bill You Do Not See in the GPU Line​

So far the math counts only the server price. Self-hosting has a second ledger:

Serving stack. vLLM (or TGI, or TensorRT-LLM) is the de facto standard. It batches concurrent requests efficiently — the reason a single card's cost amortizes across users. Benchmarks showing self-hosted vLLM at $0.065/M output vs $0.30/M managed at peak concurrency are not magic; they are batching. Set max-num-seqs and gpu-memory-utilization correctly or you will never hit the throughput numbers above.

Model weights and memory. A 70B model at INT8 needs ~70GB of VRAM plus KV-cache. The 96GB card handles it; the 20GB entry card does not. Quantization (AWQ, GPTQ, FP8) is not optional at the low end — it is the difference between fitting and not fitting. Budget time to evaluate quality at each quantization level.

What is included. Hetzner bundles power, networking (with its generous but not infinite included traffic — check the 20TB-ish allotments and €1.19/TB overage on some lines), and the physical machine. You do not pay extra for "GPU power draw" the way hyperscalers bake it into the hourly rate. That is part of why €889 looks cheap next to $6.88/hr for an H100 hour that also bundles power.

Fleet integration. If you already run CAPH, adding a GPU pool is a MachineDeployment variant with a GPU instance type, taints/tolerations, and a ClusterResourceSet for the NVIDIA device plugin. That reuses the same provisioning loop you already operate for CPU capacity. If you do not already run CAPH, the calculus is different — you are not "just adding a node," you are adopting a fleet manager.

The hidden cost is people. Someone owns driver versions, CUDA compatibility, vLLM upgrades, and model rollouts. That is a real ongoing tax that a per-token API absorbs for you. It does not appear in the €/M table. Budget it as engineering time, not euros.

Decision Framework: Own, Rent, or Hybrid​

Answer three questions in order:

1. How many tokens a month, and how bursty?

• 150M/month and >40% average utilization → self-hosted GEX131 is the cost winner, even vs cheapest APIs.

• 25–150M/month → self-hosted wins vs flagship APIs, may still lose vs cheapest open-weight APIs unless you pick a smaller model. Consider the €184 GEX44.
• <15M/month or highly bursty → hosted API. The flat card is a liability.

2. Does data leave your network matter?

If prompts contain tenant data you cannot send to a third-party proxy, self-hosting wins on privacy even when it loses on cost. Routing API keys and prompts through infrastructure you control is not a line item — it is a requirement. Hetzner's EU footprint (Nuremberg, Falkenstein, Helsinki) plus a fleet you own is a stronger sovereignty story than any amount of per-token savings.

3. How flexible is your model choice?

Hosted flagship pricing assumes you need GPT-5 / Claude Opus. If you can serve a well-tuned open model (Qwen3, Ministral, Llama) and control quality yourself, the self-hosted card's value rises. If you need the frontier model's reasoning and cannot substitute, the comparison is not apples-to-apples — you are paying for a different product.

A common hybrid that teams on an existing Hetzner fleet adopt: keep a GEX44 or GEX131 warm for baseline traffic and steady features, and overflow bursts or frontier-model calls to a managed API via a gateway (LiteLLM, Mantis) that routes by model, cost, or latency. That turns the flat card into the cheap floor and the API into the elastic ceiling, rather than forcing one to cover both.

The cheapest inference is not always self-hosted, and the cheapest API is not always cheap once volume grows. The 2026 market has settled into a clear shape: APIs win on elasticity, flat bare metal wins on sustained throughput, and the gap between "cheapest managed token" and "flagship token" is now large enough that your model choice moves the breakeven more than your GPU choice does.

If you already own the fleet, the GPU node is not a new bet — it is an extra line in a system you already operate. The question is whether you have enough steady tokens to fill it. Run the table with your own daily token counts and your own burst factor. If 40% of a card's capacity is spoken for, the €889 is not an expense. It is a discount.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. If you already run a Cluster-API-provisioned Hetzner fleet, adding a GPU node pool for self-hosted inference is the same declarative loop you use for every other capacity — no second cloud, no per-token meter on the steady load. Star the repo on GitHub or deploy your first app today.