Hetzner Already Killed the datacenter Field — What a CAPH Fleet Must Check Before August 1

Hetzner removed the datacenter property from its Servers and Primary IPs API on July 1, 2026. That's not a countdown — it already happened, a week before this post went up. A second, related removal lands August 1: the undocumented EC2-compatible routes on Server Metadata go away. If you're running a Cluster API Provider Hetzner (CAPH) fleet, both changes touch the same surface CAPH's own node bring-up depends on, and neither one announced itself with anything louder than a changelog entry.

The Timeline, in Full​

Hetzner didn't ship this as one change — it's three related deprecations on the same API surface, staged four months apart:

Two of those rows are the ones worth an actual audit right now. Here's what each one touches in a CAPH-managed fleet, and what doesn't need your attention at all.

What Actually Breaks: the datacenter Field​

The reassuring finding first, because it's not the obvious one: CAPH's own controller code never reads the datacenter field. Cloning syself/cluster-api-provider-hetzner and grepping the whole tree outside vendor/ for .Datacenter turns up zero hits. CAPH already creates every server through the Location field — it made the switch to the field Hetzner wants you on before this post was written, without anyone having to do anything. Node bring-up — the part of CAPH that actually provisions a Hetzner server and joins it to a cluster — was never at risk from the July 1 removal.

The risk lives one layer up, in hcloud-cloud-controller-manager (the CCM most CAPH fleets install via a ClusterResourceSet to get load-balancer integration and node topology labels). The CCM used server.Datacenter.Name — values like fsn1-dc8 — to populate the topology.kubernetes.io/zone label on every node. Hetzner's changelog spells out exactly what happens to a CCM that never picked up the fix:

Deployments of hcloud-cloud-controller-manager that are not updated when the field is removed from the API will panic with the following error: "Observed a panic" panic="runtime error: invalid memory address or nil pointer dereference"

That's not a hypothetical failure mode for "later" — the field has been gone since July 1, so an unpatched CCM hits this the next time it tries to sync a node. The fix shipped in v1.30.1, released February 20, 2026, which statically maps existing Hetzner locations to their legacy -dcxx zone names so the label format doesn't change for anyone already running it. Check what your fleet is actually running:

kubectl -n kube-system get deploy hcloud-cloud-controller-manager \
  -o jsonpath='{.spec.template.spec.containers[0].image}'

If that tag predates v1.30.1 (v1.30.0, v1.29.x, or older), the CCM is not reading the field it depends on anymore, and the panic in the changelog is what a node-sync attempt looks like now, not next month. The current release as of this writing is v1.33.0 (June 18, 2026) — three minor versions past the fix, so there's no reason to be running anything close to the affected range unless upgrades have been paused for a while.

Even patched, there's a gotcha worth knowing about rather than assuming away: the static mapping only covers locations that existed when v1.30.1 shipped. Any location Hetzner adds after that patch gets a bare topology.kubernetes.io/zone value with no -dcxx suffix — a different label shape than every existing node in the same fleet. If you lean on topologySpreadConstraints or zone anti-affinity rules that assume a uniform label format across nodes, a new location breaks that assumption quietly, not with an error. Hetzner's own RFC on the CCM's GitHub issue tracker (linked from the v1.30.1 changelog) proposes a longer-term fix for topology.kubernetes.io/zone and .../region, but nothing beyond the static map has shipped yet.

Upgrading the CCM Without Guessing​

If the image tag from the kubectl get deploy check above is behind v1.30.1, the fix is a version bump, not a config change — the static mapping is baked into the binary, with nothing new to set. Most CAPH fleets install the CCM through a ClusterResourceSet that references a Secret or ConfigMap holding the manifest, rather than a Helm release applied by hand, so the actual upgrade is usually: bump the image tag in that manifest, re-kubectl apply it (or let the CRS controller reconcile it if the source object changed), and roll the CCM Deployment. It runs as a single-replica Deployment in kube-system by default, so a rollout is a normal kubectl rollout restart deployment/hcloud-cloud-controller-manager -n kube-system once the manifest is updated — no node drain, no cluster-wide disruption, because the CCM doesn't sit in the pod scheduling path. Confirm it landed by re-running the same jsonpath image check and watching kubectl -n kube-system logs -l app=hcloud-cloud-controller-manager for a clean sync instead of a panic on the next reconcile loop.

What Actually Breaks: the EC2-Compatible Metadata Routes​

Same pattern, same negative result for CAPH itself. Hetzner's instance metadata service has always answered on /hetzner/v1/* — the documented route cloud-init's native Hetzner datasource uses to pull hostname, network config, and SSH keys on first boot. It turns out Hetzner's metadata service also answered AWS-shaped paths like /latest/meta-data and /2009-04-04/meta-data, undocumented and never announced until the June 30, 2026 deprecation notice that gave it a removal date. Grepping CAPH's source and the kubernetes-sigs/image-builder Packer templates it uses for Hetzner node images turns up no reference to 169.254.169.254 or any EC2-style path anywhere — CAPH-built nodes were never calling those routes.

The actual exposure is anything else running on the fleet that hardcodes an AWS-shaped IMDS path as a portability shortcut: a security or observability agent that probes 169.254.169.254/latest/meta-data/instance-id as a generic "am I on a cloud VM" check, a script ported from an AWS runbook that never got adapted for Hetzner, or a DaemonSet image built against a cloud-agnostic base that assumes EC2-style metadata everywhere. None of that is CAPH-specific — it's whatever your team layered on top of the fleet. The audit is a grep, not a code review:

grep -rn "169.254.169.254\|/latest/meta-data\|/2009-04-04" \
  --include="*.yaml" --include="*.sh" --include="*.go" .

Anything that turns up needs to move to the documented /hetzner/v1/metadata route before August 1. After that date, a hardcoded AWS-style path against a Hetzner instance returns nothing — silently, not as an error a health check would necessarily catch. The two metadata shapes aren't identical, either: the EC2 format returns flat key-value text (instance-id, hostname, public-ipv4 as separate paths), while /hetzner/v1/metadata returns a single YAML document with hostname, instance-id, region, availability-zone, public-ipv4, private-networks, and public-keys fields in one response. A tool built for the EC2 shape can't just swap the base URL — it has to parse a different response format, which is exactly the kind of thing that gets found in an incident instead of a grep if it isn't audited now.

The One Further Out: Server Types' Datacenter Fields (October 1)​

Worth one line, not a full section: the April 1, 2026 deprecation of server_types' recommended/available datacenter fields — replaced by server_types.locations.available/.recommended — removes on October 1. It only matters if something reads Hetzner's server-type recommendation data directly (a capacity-planning script, a Terraform data source) rather than going through CAPH, which doesn't touch this surface either. Same audit pattern, four months of runway instead of three weeks.

The Five-Minute Version​

If you're skimming, this is the whole audit:

• datacenter field (already removed, July 1): run kubectl -n kube-system get deploy hcloud-cloud-controller-manager -o jsonpath='{.spec.template.spec.containers[0].image}'. Below v1.30.1 → upgrade now, don't wait for the panic to show up in an alert.
• New Hetzner locations after Feb 20, 2026: check whether any node's topology.kubernetes.io/zone label is missing the -dcxx suffix that older nodes have, and audit whether anything (topology spread, zone anti-affinity) assumes a uniform format.
• EC2-compatible metadata routes (removal August 1): grep -rn "169.254.169.254\|/latest/meta-data\|/2009-04-04" across node images, DaemonSets, and init containers; repoint any hit to /hetzner/v1/metadata and its YAML response shape.
• Server Types datacenter fields (removal October 1): only relevant if something reads Hetzner's server-type recommendation data directly instead of through CAPH — same grep pattern, four months of runway.

Why This Is Your Dependency Now​

None of this required a CAPH bug to hurt you — it required nothing more than running an old CCM binary or a script someone wrote for a different cloud. That's the tradeoff of an infrastructure-provider integration you run yourself instead of one a vendor runs for you: a managed PaaS absorbs a provider's routine API cleanup on your behalf, silently, because it's their pinned dependency to track, not yours. A self-hosted Cluster API fleet makes it yours — which means "check the CCM version" and "grep for hardcoded metadata paths" belong on the same recurring maintenance cadence as apt upgrade and CVE patching, not treated as a one-time migration you do once and forget.

That's a fair trade for owning the machines instead of renting a black box, but it's a trade worth going in with eyes open — the failure mode here isn't a bug in your code, it's a changelog entry three services away from the one you actually read. Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with the same Cluster API foundation this post is auditing. Star the repo on GitHub or deploy your first app today.