Hetzner Is Building an LLM Inference API: What €889/Month of GPU and a Free Experimental Endpoint Do to the Self-Hosted AI Math

Hetzner is experimenting with LLM inference. That sentence alone was enough to put Sliplane's July 24, 2026 report at 156 points on Hacker News — not because the product is finished, but because the company behind it has spent a decade being the denominator in every "self-host on a Hetzner box" cost comparison.

If you run a Cluster API fleet on Hetzner hardware, you have probably already done the rent-vs-own math for your app servers. Now you need to do it again for inference — because the budget host that made your box cheap is testing whether it can also make your token cheap.

The early details are small but telling: an OpenAI-compatible API at experiments.hetzner.com, one model (Qwen 3 35B-A3B MoE, FP8, 262K context), no billing, no SLA, and a very Hetzner way of learning in public. Next to it sits the hardware you can already rent yourself — the GEX131 with an RTX PRO 6000 Blackwell (96 GB) at €889/month, and an entry inference box around €184/month. The question for a self-hosted PaaS is not which of those two things to pick today, but where the break-even sits between them, what sovereignty each one actually buys, and how a platform that treats inference like any other tenant workload should draw the line.

What Hetzner Actually Shipped​

Sliplane's Jonas Scholz tested the endpoint on July 23 and published the first look that HN picked up the next day. The shape is deliberately minimal:

• OpenAI-compatible API on Hetzner's own infrastructure. Create a token in the Experiments dashboard, point any OpenAI client at Hetzner's base URL, and send the same JSON you already send elsewhere.
• One model so far: Qwen/Qwen3-35B-A3B-FP8 — a 35-billion-parameter Mixture-of-Experts with 3 billion active parameters per forward pass, available as text + image in, text out, with FP8-quantized weights (~38 GB on disk, more in VRAM once you count KV cache and context). It is small enough to serve from a single high-memory GPU, large enough to be useful for real prompts.
• Experiments branding on purpose. No billing, no SLA, no production guarantees. Hetzner says it wants to learn whether people want the service, how the system scales, which features matter, and what load it can handle. Scholz calls it exactly what it is: Hetzner putting something early in front of users to see what happens, not a finished product launch.
• Speed is good, headroom is unknown. The initial test was fast, but one client at one point in time says nothing about contention when hundreds of tenants hit the same shared GPUs.
• A tutorial, not a platform. Hetzner published a short Community tutorial for wiring the endpoint into OpenCode, which underlines how little is custom — if you already speak OpenAI API, you already speak Hetzner inference.

The model itself was, in Scholz's words, "a small, slightly shitty LLM" — it followed formatting and handled an image correctly, then failed two trivial arithmetic questions. That is not the point. The point is what the experiment signals about Hetzner's strategy, and where an inference API fits in a company whose core competency is buying hardware, racking it in its own data centers in Germany, Finland, the US and Singapore, and operating it with famously thin margins.

The DIY Path You Can Already Rent​

Long before the inference experiment, Hetzner listed the hardware you would use to not need an inference API. Its public dedicated-GPU lineup in 2026 is short and legible:

Both are billed as flat monthly contracts with a one-time setup fee, no per-second meter, and no stop button. You pay for August whether you served 10 million tokens or 10 billion.

For a team already running a Cluster-API-managed Hetzner fleet, the marginal cost of self-hosted inference is that monthly line plus the operational tail: vLLM or TensorRT-LLM on the box, model weights on NVMe, autoscaling the GPU MachineDeployment separately from the CPU pool, and monitoring for the queuing that happens when a burst of agent prompts lands at 10 a.m. and the box is already saturated.

What that box can actually hold matters for the comparison. The FP8 files for Hetzner's own Qwen 35B-A3B are ~38 GB — comfortable on the 96 GB card even with a long KV cache, but the same card has no story for the 500B-750B monsters (GLM-5 at 754B, split across 8× B200s in its reference recipe) that need hundreds of gigabytes of VRAM stitched together with NVLink-class interconnects. Hetzner's public catalogue today is workstation-class GPUs, not dense B200/B300 pods. The experiment may run on different internal hardware, but if you are planning to self-host models bigger than ~70B dense or ~200B MoE, the €889 box is not the right yardstick — you are comparing against either an 8-GPU H100/B200 rental or a hosted API anyway.

Scholz's open question is exactly this: if the experiment stays at one or two small models, it remains a fun demo. If it is the first step toward a proper model catalogue backed by B200-class clusters, then Hetzner's existing advantages — its own data centers, its network, its procurement muscle, and its reputation for pricing like a wholesaler — start to compound.

The Break-Even Table: When €889/Month Beats Per-Token​

This is the core deliverable. Take three representative workloads and price them three ways: self-hosted on a GEX131, hosted inference at typical open-weight API rates, and raw GPU rental for a bring-your-own-vLLM elsewhere.

Assumptions shown explicitly so you can re-derive for your own mix:

• Self-hosted fixed cost: €889/month for GEX131 + ~€50/month for the ops tail (monitoring, weight storage, a second of SRE time amortized across the fleet) = ~€939/month all-in whether idle or hot. No per-token meter. Utilization is the variable.
• Hosted per-token (open-weight): OpenRouter-style blended pricing for 30-70B MoE models in mid-2026: $0.30–$0.60 per 1M tokens (input+output blended; some providers charge ~$0.20/$0.80 split, which nets the same range). We use $0.45 as the midpoint.
• Hyperscaler frontier API: OpenAI GPT-4o / Anthropic Sonnet-class at $2.50–$5.00 per 1M tokens blended — the price teams are actually trying to beat when they talk about self-hosting inference at all. We use $3.50.
• Raw H100 rental: $2–$3/hour on demand (~$1,500–$2,200/month if left on 24/7) or ~$1.20/hour on 12-month reservations. Same "no stop button" shape as Hetzner's box, but at 1.7–2.5× the monthly cost for ~30–50% more throughput on large models.

Read the table left to right for the takeaway most cost posts bury:

• Below ~2B tokens/month, the per-token API wins by a lot. €939 flat for 150M tokens is ~$6.77 per 1M — 15× the hosted open-weight rate. The monthly box only earns its keep when it is hot.
• The crossover for the GEX131 against a $0.45/M API is around 2–2.5B tokens/month. Against a frontier API at $3.50/M, the crossover drops to ~270M tokens/month — which is why teams serving OpenAI-class workloads see self-hosting pay back so fast.
• Against an H100 rental, Hetzner's box is cheaper at every utilization level for models that fit in 96 GB, but slower per token for very large models. The decision there is not price but whether your model needs 192 GB+ of pooled VRAM.

Sensitivity matters more than the single midpoint:

• If Hetzner prices its eventual inference API at its usual wholesale margin — say $0.25–$0.35/M for a Qwen-class model — the self-hosted crossover pushes out to ~3–4B tokens/month. At that price the managed endpoint is the rational default for everyone except the heaviest tenants, which is exactly the utilization story Scholz flags: a shared API can keep a GPU busy across hundreds of low-volume tenants in a way a per-tenant GEX131 cannot.
• If your workload is bursty (the agent that hammers the model at standup and then sleeps), the flat box pays for idle hours the API does not. A 5× morning burst on a GEX131 means either queueing or provisioning a second €889 box that sits idle 20 hours a day. The API absorbs it for $0.45/M and no idle tax — the same flexibility gap that makes metered builders attractive for git-push bursts, but in reverse for steady app traffic.

For a PaaS already on Hetzner hardware, the honest framing is therefore not "owning is cheaper" but "owning is cheaper above a throughput line that depends on which API price you are actually beating." Draw that line for your own token mix before you buy the GPU.

Sovereignty: Does a Hetzner-Managed Endpoint Change the Argument?​

Some teams self-host inference for cost. Others self-host because the prompt contains the thing they cannot send to a US hyperscaler.

Hetzner's experiment matters differently for those two groups.

What self-hosted vLLM on a Hetzner box gives you today: data stays on a server you rented, in a Hetzner data center, in an EU jurisdiction, operated by an EU-headquartered company. No US provider sees the prompt, the completion, or the embeddings. For GDPR, for CADA-aware public procurement, and for the regulated buyer who needs more than a region code, that is the strongest posture short of on-premises.

What a Hetzner-managed inference API would give you at its best: the same physical and corporate jurisdiction — EU data centers, EU operator — but with the operational boundary moved from your MachineDeployment to Hetzner's internal inference fleet. Your prompt still stays inside Hetzner's network, not inside OpenAI's or Anthropic's. That is a materially better sovereignty story than any US-hosted API, even one served from an eu-west region, because the jurisdiction question is about who operates the company, not just where the rack is.

It does not, however, collapse to the same guarantee as running the weights yourself:

Two practical consequences follow:

1. For GDPR-strict but not certification-strict workloads — internal agents, support copilots, retrieval over tenant data — a Hetzner-priced, EU-operated inference endpoint is plausibly "sovereign enough" for the same buyer who already trusts Hetzner with their app servers. The cost-vs-sovereignty tradeoff that drives teams to vLLM today gets weaker if the managed price is low and the jurisdiction is the same.

2. For certification-strict workloads — government, healthcare, defense — even Hetzner's EU operator story may not clear the bar on its own. OVHcloud's June 2026 SecNumCloud-qualified offering is instructive: France's ANSSI standard requires documented control separation and vetted operational processes, not just an EU legal entity and an EU data center. If your procurement needs a named qualification, "we run on Hetzner hardware" and "we call Hetzner's inference API" both need to point at a specific scheme — otherwise you are still in the jurisdiction argument, not the certification argument.

In short: a Hetzner inference API would expand the set of workloads that can stay EU-sovereign without running their own GPUs, but it does not replace the self-hosted path for tenants who need weight choice, log control, or a formal sovereign qualification.

Where Inference Sits in a PaaS That Treats It Like Any Workload​

Bex's design constraint is useful here because it is deliberately boring: inference is not a separate product with a separate control plane. It is a tenant workload that happens to request nvidia.com/gpu: 1 and a RuntimeClass that knows about the right drivers. The same Cluster API fleet that bin-packs web services and workers also bin-packs inference — GPU MachineDeployments scale like any other node pool, and an App CR does not care whether the container inside it is a Next.js frontend or a vLLM server.

That view gives you a clean decision tree for picking the path per app, not per platform:

Does the prompt contain data that cannot leave your tenancy's GPU?
  yes → self-hosted vLLM on a GEX-class node in the same fleet
  no ─┬─ Is steady throughput > ~2B tokens/month at open-weight pricing
      │   (or > ~300M at frontier pricing)?
      │     yes → self-hosted is cheaper; keep the workload in-fleet
      │     no  → managed API is cheaper; the flat box will idle
      └─ Is the workload bursty (5×+ peak/avg)?
            yes → API absorbs bursts; add elastic inference via the
            │     managed endpoint for the top of the curve, keep a
            │     small GPU pool for the base load
            no  → either path works; pick on sovereignty and weight
                  choice, not cost

Three operational details make that tree more than a slide:

• Separate the GPU node pool. Do not schedule vLLM onto the same mixed pool as your CPU services. A dedicated MachineDeployment with taints (nvidia.com/gpu:NoSchedule) and a ClusterAutoscaler that scales the GPU pool on pending GPU pods keeps inference from starving web traffic and lets you measure the GPU pool's cost in isolation — the same per-tenant showback that opencost-style exporters already give you for CPU.

• Do not hide a second PaaS inside the first. If you add a hosted inference provider, consume it as a tenant-level secret and URL (an Inference CR that holds the base URL + API key), not as a platform-level dependency that every app must be rewritten around. The tenant that wants self-hosted vLLM tomorrow should be able to swap the endpoint without redeploying the platform.

• Keep the exit path real. Hetzner's experiment has no billing and no SLA today. Even if it graduates to a priced product, the platform's value is that a tenant can leave the API for their own box with a one-line change — the same portability that makes a git-push PaaS credible against a closed PaaS's repricing in the first place. The June 2026 cluster of repricings across Render, Vercel, Fly, Docker Hub, Hetzner itself, and OVHcloud is a reminder that every layer of a rented stack can move at once; a workload that can run on an API or on an owned GPU is hedged against whichever layer moves next.

What to Do With an Experiment​

Do not move production inference to experiments.hetzner.com this week. Hetzner itself says not to — no billing, no SLA, one model, undocumented flags like enable_thinking that may change, and an explicit goal of learning rather than serving.

Do treat the experiment as a credible signal. A company that has spent years being the cost floor for self-hosting does not test managed inference lightly — the DRAM squeeze that drove Hetzner's own 30–170% price hikes in the first half of 2026 is the same squeeze that makes denser, better-utilized GPU fleets worth building, and an inference API is exactly the product that turns spare or planned GPU capacity into shared, billable utilization.

The practical move for a team on a Hetzner-backed PaaS is threefold: keep the vLLM-on-GEX131 path warm (pin a working vLLM image + App template so you can land a tenant's inference workload on owned hardware in minutes), track what Hetzner does with model catalogue and hardware behind the experiment (the jump from one Qwen to a catalogue served on B200-class pods is the real tell), and run the break-even for your own token mix at $0.30, $0.45, and $3.50 per million so you know which side of the line you are on before the pricing page goes live.

Self-hosting's pitch was never that every workload belongs on your box. It was that the box you own should be the cheaper, more sovereign place for the workloads that already live there — and that any workload that leaves the box should be able to come back without a rewrite. A Hetzner inference API, if it graduates, does not break that pitch. It just moves the line where it starts being true for inference — and makes the platform's job, as ever, to keep both sides of that line runnable on the same fleet.

Hetzner Inference is experimental as of July 2026 — free, one model, no SLA. The GEX131 (RTX PRO 6000, 96 GB, €889/month) and GEX44 (~€184/month) prices are Hetzner's public dedicated-GPU list as of August 2026. Token benchmarks and the $0.45/M open-weight / $3.50/M frontier assumptions are 2026 market midpoints — re-derive them for your own model mix.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Inference included: run vLLM on a GPU node in the same Cluster-API fleet, or point an app at any OpenAI-compatible endpoint without changing the deploy. Star the repo on GitHub or deploy your first app today.