Hetzner Is Giving Away Free Inference. Should Your Fleet Still Run Its Own vLLM?

The budget bare-metal host just became a model provider. In August 2026, Hetzner opened a free, OpenAI-compatible inference API serving open-weight models from its own infrastructure — no GPU server to provision, no vLLM to tune, just a token from experiments.hetzner.com and a base-URL swap. For a self-hosted fleet that planned to run its own inference on Hetzner GPU iron, that raises an uncomfortable question: is the vLLM box you were about to build already obsolete?

Short answer: use the free API now, keep the vLLM plans. The longer answer is a rent-vs-own calculation where the "rent" side currently costs zero — and the catch is everything that isn't price. This post runs the numbers, reads Hetzner's own first-week retrospective for what "free" actually guarantees, and gives you a decision checklist for sequencing your inference roadmap.

What Hetzner actually shipped​

The facts, minus the hype. Hetzner's Experiments Platform (experiments.hetzner.com) gained an Inference offering: an OpenAI-spec API at inference.hetzner.com/api/v1, token-gated through an Inference tab, free while experimental, explicitly without an SLA, and subject to change or discontinuation.

It serves open-weight models only — Qwen 3.6-35B-A3B as the baseline, plus DeepSeek, GLM, and Kimi variants for the larger end of the catalog — with the Qwen endpoint advertising a 262K-token context window. On privacy, Hetzner's docs state it keeps only usage-tracking data (timestamps, token counts) and does not store request or response content.

Two details matter more than the model list. First, the timeline: Hetzner quietly enabled the new models on August 7, 2026 and announced them on August 10. Within hours, word of mouth drove request volume past the company's most optimistic expectations.

Second, the hardware split behind the API: the smaller Qwen model runs on GEX131 single-GPU servers (one RTX PRO 6000 Blackwell each), while the large DeepSeek, GLM, and Kimi models each got dedicated custom 8-GPU servers. Hetzner serves this stack with vLLM, llm-d request routing, KV-cache offloading into host RAM, and Open WebUI — the same open-source serving stack you would assemble yourself, operated at a scale you won't match on a single box.

Compatibility is the quiet killer feature. Because the endpoint speaks the OpenAI chat-completions dialect — SSE streaming, standard max_tokens, image_url vision parts — existing clients work with a base-URL change and no new adapter code. Community tooling arrived within days: a hetzner-inference toolkit with LiteLLM, Open WebUI, and Aider configs, provider entries in tools like kprompt and clai, and an official Hetzner community tutorial for OpenCode. If your agent sandboxes already talk to an OpenAI-compatible gateway, pointing them at Hetzner is a config edit, not a project.

The rent-vs-own math (with the volume knob, not one data point)​

Here is the core deliverable: a break-even comparison between renting Hetzner's API and owning a vLLM box on Hetzner iron, across three realistic traffic levels. Assumptions are stated so you can re-run them with your own numbers.

The own side. A Hetzner single-GPU dedicated box in the GEX44 class (RTX 4000 SFF Ada, 20 GB VRAM) is community-reported at roughly €184/month — call it ~$200/month all-in, since Hetzner bundles power. That buys you a fixed, always-on inference server with no per-token meter and no rate limiter except your own VRAM.

The honest unit-cost anchor for self-hosted vLLM serving comes from NVIDIA-cited SemiAnalysis benchmarking: about $0.09 per million tokens for a large open model on H100-class hardware via vLLM. Your 20 GB card won't match H100 throughput, but it sets the order of magnitude: owned inference costs cents per million tokens if you keep the box busy.

The rent side. Today: $0 per million tokens, fair-use rate limits, no SLA. But "free while experimental" is a price, not a promise — so the table also prices the plausible paid future using the going rate for hosted 32B-class models (Cloudflare Workers AI lists its 32B Qwen-class models at $0.66 input / $1.00 output per million tokens; blended ~$0.75/MTok is a fair planning figure).

One capacity assumption sits under the own column: a ~$200 box sustains roughly 120 tokens/second on a 14B-class model, or about 310M tokens/month at full tilt. Past one box's ceiling you add boxes, so the own side stops getting cheaper per token — the high row is three boxes, not one magically faster one. (This is also why the breakeven here lands far below the ~50M-tokens/day figure quoted for H100-class 70B serving: smaller card, smaller model, smaller crossover. Every breakeven number is tier-specific.)

Three readings fall out. At prototype volumes, renting wins at any plausible paid price — a fixed box serving 50M tokens/month pays ~$4/MTok in effective unit cost while sitting idle. Near ~250–300M tokens/month, one owned box breaks even against paid-API rates, but with no headroom left — treat that as the trigger to plan capacity, not to celebrate it. Past a billion tokens, small-box owning reaches rough parity rather than a discount; a real per-token discount needs denser iron (bigger cards, better dollars-per-throughput), which is a larger capex decision than the headline €184 box.

But price is only half the own-side equation. The other half is a VRAM ceiling that no cost math can negotiate with:

This is the crux: the ~€184 box cannot serve the 35B-class models Hetzner gives away for free. Matching the free API's catalog on your own iron means GEX131-class hardware or better — a different capex line entirely. So the real choice isn't "free API vs. cheap box." It's "free API vs. a materially more expensive box than the headline Hetzner price" — which pushes the break-even volume for catalog parity well above the table above.

What week one revealed about the word "free"​

Hetzner published a candid retrospective after the first week, and it reads as a field guide to what free-tier inference guarantees in practice. Demand exceeded the company's most optimistic expectations within hours of the August 10 announcement. Token throughput held up, but latency degraded: 99th-percentile time-to-first-token sat at 5–10 seconds for much of the time, with short spikes.

General rate limits were adjusted downward to spread capacity, some usage patterns were filtered as malicious, and the OpenClaw tab was disabled for accounts that hadn't signed in because the small Kubernetes cluster running OpenClaw instances hit capacity.

Then the tell: after one week, Hetzner announced the experiment would continue "on a smaller scale, with smaller models" while it makes the improvements large-model demand requires. The large models were the draw; the large models are what got throttled first.

None of this is a complaint — it is exactly what "experimental, no SLA, may change or be discontinued" means, honored in public. But it defines the availability shape you are building on: generous when idle, degraded or narrowed when popular, with model availability as a capacity-management lever rather than a contract.

A prototype tolerates that shape. A production agent fleet with latency SLOs does not. And one quieter constraint: Hetzner couldn't ship Kimi K3 at all — its license terms didn't permit a free experiment — a reminder that the free catalog is filtered by licensing lawyers as well as VRAM.

Why Hetzner bypassed its own agent to ship the API​

The retrospective's most interesting thread isn't the hardware; it's the origin story. The Experiments Platform started four months earlier with OpenClaw — a preconfigured AI-agent instance, with Hetzner's inference API tucked inside it, available only through the agent so request volume stayed controllable.

Two things went wrong with that plan. Small models like Qwen 3.6 proved sufficient for most OpenClaw use cases, which was good news — and OpenClaw's upstream code quality was, in Hetzner's words, "a constant problem," with each release introducing new bugs to absorb.

So Hetzner cut the agent out of the loop and offered inference directly as an OpenAI-spec API. That decision is the architectural lesson for every self-hosted platform watching: the durable interface is the model endpoint, not the harness. Agents churn — weekly releases, shifting config schemas, quality variance.

The chat-completions API is stable enough that a whole ecosystem (LiteLLM, Open WebUI, Aider, OpenCode, kprompt) can standardize on it, which is precisely why community integrations for Hetzner's endpoint appeared in days rather than quarters.

For your own roadmap, the implication runs both directions. Consuming inference through an OpenAI-compatible gateway (LiteLLM or equivalent) is what makes Hetzner's free tier a zero-cost experiment and a zero-cost exit — the day the experiment narrows, ends, or prices up, you repoint the gateway at your own vLLM or another provider. And if you ever serve inference yourself, OpenAI-compat is what makes your endpoint adoptable. Hetzner learned this by operating both sides in public.

The decision checklist: rent, own, or sequence both​

Most self-hosted teams shouldn't pick one side permanently. They should sequence: rent now, own when a trigger fires. Use this checklist:

Rent Hetzner's API while ALL of these hold:

• Traffic is bursty or prototype-scale (well under ~250M tokens/month sustained)
• No latency SLO exists that a 5–10s p99 TTFT would violate
• The catalog's models (and their licenses) cover your agent workloads
• EU-based processing with no stored request/response content satisfies your data posture
• You access it behind your own gateway, so the provider is one config line

Build the owned vLLM when ANY of these fires:

• Sustained volume crosses break-even (~250–300M tokens/month vs. paid-API rates)
• You need a model the free catalog excludes (license-filtered like K3, fine-tuned, or larger than the scaled-down tier)
• Rate limits or TTFT spikes have paged you — or would violate a tenant-facing SLO
• The experiment's terms change: pricing appears, the catalog narrows further, or the service is discontinued
• Data policy requires inference on iron you control, not just in-region iron you rent per token

Note what "build" means after the VRAM table: catalog parity with the free tier starts at 96 GB-class GPUs, not the €184 box. Budget the GEX131 tier (or 48 GB cards with aggressive quantization and short contexts) and validate fit with your actual context lengths before committing — a 35B model at 262K context has a KV-cache appetite that dwarfs its weights, which is exactly why Hetzner offloads KV cache to host RAM.

What this means for a self-hosted PaaS roadmap​

Step back from the price war framing, because the strategic signal is bigger than one free endpoint. The budget host validated the exact stack a self-hosted platform would run — vLLM, open weights, OpenAI-compat serving, KV-cache offload — and proved demand for it at a scale that surprised even Hetzner. That de-risks the "own" side of your roadmap: you are no longer betting that open-weight serving works, only deciding when your volume justifies operating it.

The sequencing practically writes itself. Phase one: put a LiteLLM-style gateway in front of all agent and sandbox model traffic today, with Hetzner's free endpoint as the default upstream. You get zero-VRAM-capex model access for prototyping while your traffic pattern reveals its real shape — volume, burstiness, context lengths, latency sensitivity.

Phase two: when the checklist fires, stand up owned vLLM capacity behind the same gateway and shift traffic without touching a single agent config. The gateway is the roadmap; the providers behind it are interchangeable capacity.

Hetzner also set the competitive bar for what "good enough" hosted inference looks like from infrastructure you already trust: EU-based, OpenAI-compatible, no content retention, honest about limits. Any owned stack you build later has to clear that bar on reliability and model choice, because your tenants will compare it against a free tier that already exists. That is a healthier forcing function than any internal spec.

The budget host giving away inference isn't the end of self-hosted vLLM plans. It's a free, time-boxed gift of the hardest thing in capacity planning: real demand data. Take the gift, instrument behind a gateway, and let measured traffic — not hype or fear — decide when the GPU invoice is worth signing.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.