Hetzner Gives Away the Model Calls: What Free Inference Is Actually Worth to an Agent Builder

Your hardware vendor just zeroed out a line item on your AI bill. In July 2026, Hetzner — the German hoster famous for dedicated servers priced like typos — launched an experimental Inference API that serves open-weight models through an OpenAI-compatible endpoint, free of charge while the experiment runs. If you already rent the box next door for your agent sandboxes, the model calls are now the cheapest part of the stack: they cost nothing.

The number up front: for a typical solo builder running about five serious agent sessions a day, Hetzner's free endpoint replaces roughly $30 a month of hosted open-weight inference — or about $13 a month against the cheapest frontier tier, with a catch that flips that comparison once your prompts grow. Here is the worked table; receipts and caveats below.

That last row is the point the comparison tables miss. Self-hosting a 35B-class model on your own Hetzner GPU server starts at four figures a month before you serve a single token. Hetzner is now offering to eat that serving cost entirely — on its own hardware, in its own EU data centers — while it validates the platform. The only question is what "while experimental" forbids you from depending on.

What Hetzner actually ships​

The API went live on July 24, 2026 as part of Hetzner's Experiments platform, its space for testing products it may eventually sell. You create a token at experiments.hetzner.com/inference, point any OpenAI-compatible SDK at https://inference.hetzner.com/api/v1, and send standard /v1/chat/completions requests. Three endpoints exist: /v1/models, /v1/completions, and /v1/chat/completions. No GPU to rent, no weights to download, no vLLM to babysit.

The model shelf is small and explicitly unstable. As of this writing the docs list two models, both Apache 2.0 Qwen weights with 262,144-token context windows and vision input: the MoE Qwen/Qwen3.6-35B-A3B-FP8 (35 billion total parameters, ~3 billion active per token) and the dense Qwen3.8-27B. Hetzner says the selection "will change during the experimental phase" and that the /v1/models response is definitive — check it at runtime, not in documentation.

The limits, per API key, from the current official docs:

Exceed any of them and you get HTTP 429. Note what is missing: the August snapshot of these same docs also capped usage per 24 hours (500M input / 5M output tokens a day); the current table lists only 60-second windows. The experiment visibly moves. Do not hard-code assumptions about it.

Two privacy facts matter for EU builders. All processing happens in Hetzner's German and Finnish data centers, and Hetzner states it stores only usage metadata (timestamps, token counts — the telemetry a future billing system needs) while the content of requests and responses is not stored and there are no plans to store it. Its stated intention: use its own data centers and hardware to give customers affordable access to open-weight models.

Early independent numbers suggest the endpoint is genuinely fast, not a throttled teaser. Sliplane.io's first look measured 153 ms median time-to-first-token and 224 output tokens per second — with the correct caveat that a single-client snapshot is not an SLA.

The cost math, worked end to end​

Agent workloads burn tokens in a shape nothing else does: every turn replays the accumulated context, so input dwarfs output. A Vantage analysis of 50-turn coding sessions puts one serious session at roughly 1 million input tokens and 40,000 output tokens, a 25:1 ratio. An April 2026 arXiv study of agentic coding tasks found an average of 4.17M tokens per task at $1.857 — and warned that runs on the same task can differ by up to 30×, with input tokens driving the bill. Take the Vantage session as the "typical serious session" and the arithmetic is straightforward.

Five such sessions a day, 22 working days a month: 110 sessions, 110M input tokens, 4.4M output tokens.

Against hosted open-weight pricing for the 30B class — Qwen3-32B currently lists between $0.08 and $0.36 per million input tokens across gateways, with Groq at $0.29 in / $0.59 out and comparable Together tiers nearby — a representative $0.25/M input and $0.60/M output gives 110 × $0.25 + 4.4 × $0.60 = $30.14 a month. That is the honest like-for-like number: same weight class, metered.

Against the cheapest frontier tier the math is subtler. Claude Haiku 5.5 and GPT-6 Luna both launched at $0.10/M input and $0.50/M output in October 2026 — which prices the same workload at roughly $13 a month. But those are short-prompt rates.

Haiku 5.5 jumps to $0.50/$2.50 once a prompt crosses 100,000 tokens, and agent prompts with tool schemas and file context cross 100K routinely. At over-100K rates the same workload costs about $66 a month. Hetzner's flat $0 with a 262K context window and no tier jump looks very different once your agent's context grows past the cheap tier's cliff — which is exactly where agents live.

Sensitivity across usage levels (22 working days, representative open-weight pricing):

*Short-prompt rates; over-100K prompts roughly quintuple the frontier column.

One more honest caveat before you celebrate: a 35B MoE with 3B active parameters is not frontier quality. It is a capable small model — fine for triage, extraction, drafting, and many coding subtasks — but it will not match Opus- or Sonnet-class reasoning on hard tasks. The $30 comparison is the fair one (same weight class, metered); the frontier column answers "cheapest capable alternative," not "equal quality." Free inference that cannot do the task is not savings.

And the self-hosted row: serving these weights yourself on Hetzner hardware starts with the GEX63 at €999 a month plus a €999 setup fee (October 2026). A 35B FP8 model needs on the order of 35GB for weights alone, so anything smaller than a big-VRAM card is not in the conversation. The free endpoint collapses a four-figure fixed cost to zero — which is precisely why the fine print deserves its own section.

The fine print: what you may not depend on​

Every experimental constraint below maps to one concrete thing it forbids. Read them as a list of load-bearing walls you cannot lean on.

The 10-requests-per-minute cap is the binding constraint. The token budgets are generous — 4M input tokens per 60 seconds covers even context-heavy agent turns, and 100K output tokens per minute is roughly 7× headroom over continuous full-speed generation at the measured 224 tok/s. But 10 requests per 60 seconds means one request every 6 seconds, sustained.

A tool-calling agent in an active loop routinely fires faster than that during bursts: read file, run command, read output, repeat. Bursty loops will eat 429s. Single-threaded interactive use fits; parallel agents, CI-driven fleets, and tight tool loops do not. If your loop cannot back off and retry on 429, do not point it here.

The model list can change under you. Hetzner says so directly: the selection changes during the experiment and /v1/models is definitive. A second model (Qwen3.8-27B) already appeared after launch; models can equally disappear. Never hard-code a model id without a fallback, and never promise a user a specific model's behavior.

There is no SLA, no backup, and no production blessing. The docs state performance and availability are not guaranteed, especially under high demand, and explicitly say not to use the platform for production environments. Treat every call as best-effort.

There are no published future prices. The API is free while experimental, with advance email notice before billing begins — but nobody outside Hetzner knows what the metered rates will be. Any TCO model with a horizon past "the experiment" has a blank cell where the inference row goes. Budget the hosted-equivalent $30-ish as your fallback, not zero.

There is no DPA. Hetzner's no-content-storage posture is genuinely good, but regulated EU workloads need a signed Data Processing Agreement before personal data flows to any third-party API, and none is publicly documented for the experimental endpoint. Prototype with synthetic data; keep regulated data on your own boxes until the paperwork exists.

Use it like this, not like that​

The sanctioned shape is development, not production: run your dev and staging agent loops against it, evaluate whether a 35B-class open model clears your quality bar before you commit to a paid provider, and iterate on prompts, tools, and harnesses at zero marginal cost. Those are exactly the workloads where downtime costs you minutes, not customers.

Keep the integration portable, because the experiment will end or change. The whole adoption is two config values — base URL and key — so keep them that way, and add a paid fallback for 429s and outages:

from openai import OpenAI
 
primary = OpenAI(
    base_url="https://inference.hetzner.com/api/v1",
    api_key="YOUR_HETZNER_TOKEN",  # free while experimental
)
fallback = OpenAI(api_key="YOUR_PAID_KEY")  # metered, SLA-backed
 
def complete(**kwargs):
    try:
        return primary.chat.completions.create(
            model="Qwen/Qwen3.6-35B-A3B-FP8", **kwargs
        )
    except Exception:
        kwargs["model"] = "your-paid-30b-class-model"
        return fallback.chat.completions.create(**kwargs)

Query /v1/models at startup, retry with backoff on 429, and fail over to a metered provider with the same OpenAI-compatible interface. Your application code never changes; only the config does. That is the entire migration story, and it is why experimenting here is nearly free even counting your time.

Your hardware vendor is now your inference vendor​

Step back and notice what changed structurally. The classic self-hosting bill has two parts: a fixed part (the rented box, a Hetzner dedicated server from around €44 a month for something like an AX42-class machine) and a variable part (metered inference from a US API vendor, scaling with every agent loop you run). Hetzner just offered to zero the variable part for small-model workloads — served from the same EU data centers, on hardware it already owns, with no content retention.

No PaaS comparison table prices this in, because every comparison table assumes inference and compute come from different vendors with different meters. When one vendor owns the building, the machines, and now the model endpoint, the bill collapses to a flat rental plus zero — at least while the experiment runs. Even if metered billing arrives later, Hetzner's stated goal of "affordable access to open-weight models" on owned hardware points at cost-plus pricing from a company famous for undercutting cloud GPU margins, not SaaS multiples.

The strategic read is simple: inference is becoming a feature of the building, like bandwidth. Hetzner is testing whether the company that rents you the rack can also sell you the tokens — and right now, the tokens are free. Use them to find out what your agents can do on a 35B-class model, keep your paid fallback warm, and enjoy the brief historical window where the meter runs at zero.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.