Hetzner Removed the EC2-Compatible Metadata Routes on August 1 — Every Bootstrap Script Still Curling /latest/ Is Now Broken

The cluster was healthy an hour ago. Then an autoscaler replaced one node, and the new machine came up named ubuntu, joined with the wrong provider ID, and sat at NotReady while its replacement did the same thing. Nothing in your repo changed. Nothing in your Kubernetes version changed.

What changed is that on August 1, 2026, Hetzner Cloud removed the EC2-compatible routes from its Server Metadata service — the /2009-04-04/ and /latest/ paths that had quietly answered alongside the documented /hetzner/v1/* ones for years. The deprecation notice went up on the Hetzner Cloud API changelog on June 30, 2026, with the terse note that the routes "will be removed after 1 Aug. 2026." They were. If you have a provisioning script that does curl -s 169.254.169.254/latest/meta-data/instance-id, it now returns a 404, and whatever it was feeding — a hostname, a node label, a provider ID — is now an empty string.

We flagged this deadline in July, alongside the datacenter field removal, while there was still runway. This post is the other half: the deadline has passed, so what follows is not a warning but a migration reference — the route-by-route mapping, the response-shape changes that produce wrong values instead of errors, and the diff.

Here is the one-line audit, before anything else:

rg -n --hidden -g '!.git' \
  -e '169\.254\.169\.254' -e '/latest/meta-data' -e '/latest/user-data' -e '/2009-04-04' \
  -e 'datasource_list' -e 'ds\.meta_data'

Run that across your infrastructure repo, your Terraform modules, your Ansible roles, and your Cluster API manifests. Every hit on /latest/ or /2009-04-04 is broken today. Every hit on 169.254.169.254 needs the table below.

The migration table​

The replacement is not a rename. Some routes map one-to-one, some change response shape, and some have no equivalent at all — which is the part that turns a five-minute sed into an afternoon.

Three gotchas hide in that table, and each one produces a silent wrong value rather than a loud failure.

user-data becomes userdata. The hyphen is gone. A script that pattern-substitutes /latest/ → /hetzner/v1/ and calls it done will get a 404 on the one route that was carrying your bootstrap payload.

public-keys changed arity. On EC2 you fetched a single key by index and got an ssh-rsa AAAA… string. Hetzner returns the whole set. Anything that piped the old route straight into authorized_keys will now write a serialized array into that file, which does not fail — it just means the key no longer matches and you are locked out of a node you cannot debug.

The root document is YAML; the sub-paths are plain text. curl 169.254.169.254/hetzner/v1/metadata returns a YAML document with hostname, instance-id, public-ipv4, network-config, public-keys, and vendor_data at the top level. curl 169.254.169.254/hetzner/v1/metadata/instance-id returns the bare value with no quoting. If your script fetched the root and used jq, it never worked in the first place and you should reach for yq.

And note what has no replacement: there is no instance-type, so any node-labelling logic that derived a machine class from metadata has to get it from the Hetzner API (with a token) or from your machine template instead. There is never an iam/security-credentials route, because Hetzner has no instance-role concept — which, incidentally, means a Hetzner metadata endpoint has always been a much less interesting SSRF target than an AWS one.

The 20-minute audit​

The idiom hides in four places. Check all four; they fail at different times, which is why partial fixes look like they worked.

1. cloud-init datasource pins. Grep for datasource_list in /etc/cloud/cloud.cfg.d/ and in any golden-image build. A pin of datasource_list: [Ec2] — common in "works on any cloud" image recipes — is now fatal on Hetzner. Unpinned or [Hetzner] is fine.

2. preKubeadmCommands / postKubeadmCommands in KubeadmConfigTemplate and KubeadmControlPlane resources. This is the highest-risk location: it runs exactly once, at node join, on a machine nobody is watching.

3. Monitoring and agent auto-detection. Datadog, Telegraf's aws_ec2 processor, and similar agents probe 169.254.169.254 EC2-style to derive a hostname or cloud tags. These degrade quietly: you get a different hostname than yesterday, so your dashboards show a fleet of new hosts and your old ones go silent. Pin the hostname explicitly (hostname: in datadog.yaml) rather than letting metadata detection decide.

4. Terraform / Ansible user_data templates. The script is in a template file, not in the resource, so terraform plan shows nothing. It only surfaces on the next instance replacement.

To verify on a live node:

for p in instance-id hostname availability-zone region public-ipv4; do
  printf '%-20s %s\n' "$p" "$(curl -fsS --max-time 5 \
    http://169.254.169.254/hetzner/v1/metadata/$p || echo '<FAILED>')"
done

If any line prints <FAILED> on a node that is otherwise networked, you have a different problem — link-local routing. The metadata service is reachable from the host network, not from an arbitrary pod; if you need it from inside a pod, that pod needs hostNetwork: true or an explicit CNI allowance.

Why cloud-init survived and your curl didn't​

The asymmetry here is the actual lesson, and it is not about Hetzner.

cloud-init's DataSourceHetzner was never affected, because it never used the EC2-compatible routes. It hardcodes BASE_URL_V1 = "http://169.254.169.254/hetzner/v1" and derives metadata_url and userdata_url from it, with 60 retries at a 2-second timeout and a 2-second wait between attempts. ds-identify selects it from DMI data at boot, so an image with an unpinned datasource list picks Hetzner on a Hetzner machine without being told. That code is versioned, tested against the provider it names, and gets a deprecation cycle's worth of warning through the distro.

The curl in your preKubeadmCommands block has none of those properties. It is a string inside a YAML field inside a CRD. It has no retry, no timeout, no error handling — curl without -f exits 0 on a 404 and prints the error body to stdout, so the failure arrives as a value, not as a non-zero exit. Nothing in CI executes it. And it runs once, at node join, when there is no human attached.

That is the whole story of why a change announced a month in advance still produced a surprise outage: the announcement reached the maintainers of cloud-init and the Hetzner Terraform provider, and it did not reach the six-line shell snippet you copied from a 2021 blog post about AWS.

The fix, as a diff​

The literal repair is small:

 preKubeadmCommands:
-  - INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
-  - HOSTNAME=$(curl -s http://169.254.169.254/latest/meta-data/hostname)
+  - INSTANCE_ID=$(curl -fsS --retry 5 --retry-connrefused --max-time 10 \
+      http://169.254.169.254/hetzner/v1/metadata/instance-id)
+  - HOSTNAME=$(curl -fsS --retry 5 --retry-connrefused --max-time 10 \
+      http://169.254.169.254/hetzner/v1/metadata/hostname)
+  - test -n "$INSTANCE_ID" || { echo "metadata fetch failed" >&2; exit 1; }
   - hostnamectl set-hostname "$HOSTNAME"

The -fsS matters as much as the path change. -f makes a 404 a non-zero exit, -S still prints the error, and the explicit test -n turns an empty value into a failed bootstrap instead of a node named after the empty string. A node that fails loudly at join gets replaced by the machine controller; a node that joins with garbage metadata stays in your cluster and confuses you for a week.

The better fix is to not curl at all. Cluster API's kubeadm bootstrap provider already exposes cloud-init's parsed metadata to templates, so the hostname you were shelling out for is available directly:

preKubeadmCommands:
  - hostnamectl set-hostname "{{ ds.meta_data.hostname }}"

That expression is resolved by cloud-init from the datasource it already selected, which means it follows the provider's supported path and inherits the retry behavior. It also removes the metadata endpoint from your list of node-join dependencies entirely. If you can express what you need as ds.meta_data.*, do that; reserve raw curl for the fields cloud-init does not surface, and wrap those in the fail-loud form above.

Compatibility shims have expiry dates​

The generalizable point: an "EC2-compatible" endpoint is a migration aid with an expiry date, not an API contract. Hetzner never documented those routes as supported. They existed so that tooling built for AWS would boot on Hetzner during a migration — and once the ecosystem's serious consumers (cloud-init, the Terraform provider, the CCM) had first-class support, the shim was carrying nothing but copy-paste. Removing it was the correct call. The code that broke was code that had promoted an undocumented convenience into a dependency.

That pattern recurs everywhere: S3-compatible object storage APIs that implement 80% of the surface, Redis-compatible key-value stores, Postgres wire-protocol-compatible engines. Each is genuinely useful, and each has a boundary that is discovered rather than published. The defense is not to avoid them — it is to know, for every external endpoint your bootstrap path touches, whether it is in the vendor's documented surface. If it is not, it belongs in a list with a review date.

While you have the repo open, two more Hetzner deadlines land inside the next eight weeks, and they were announced on July 8 — after our earlier audit post covered the datacenter and server-type removals, so they are not on that list:

Neither is a compat shim — both are ordinary deprecations of documented surface — but they live in the same manifests, so do them in the same pass. The expensive part is never the fix; it is the context switch back into infrastructure code you have not opened since you wrote it.

The uncomfortable truth about self-hosting on a single provider is that your platform's uptime now includes that provider's deprecation calendar, and nobody is going to page you about it. Subscribe to the changelog feed, and treat every "removed after <date>" line as a ticket rather than an email.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, on Hetzner or anywhere else. Star the repo on GitHub or deploy your first app today.

Sources​

• Hetzner Cloud API Changelog — EC2-compatible Server Metadata route deprecation (June 30, 2026) and removal (August 1, 2026); datacenter endpoint removal, DNS ttl, dns_ptr, and assignee_type changes.
• Hetzner Cloud API Reference — Server Metadata — documented /hetzner/v1/* routes and response shapes.
• cloud-init DataSourceHetzner — BASE_URL_V1, retry and timeout constants.
• Cluster API Provider Hetzner (CAPH) — preKubeadmCommands patterns and metadata-service reachability.
• hcloud-cloud-controller-manager — first-class metadata consumption in the node controller.
• Sample hetzner/v1/metadata document — top-level keys and network-config structure.