Hetzner DNS Makes `ttl` Mandatory: Audit the One ACME Call That Can Break Renewal

On September 30, a previously tolerated omission becomes an API error in Hetzner Cloud DNS. That is exactly the sort of change that can hide in a dependency upgrade, appear during an unattended certificate renewal, and turn into an HTTPS incident only when the current certificate is close to expiry.

There is an important qualification before anyone starts rotating DNS credentials or rewriting every ACME integration: this is not a blanket change to all DNS record updates. Hetzner's July 8 changelog applies it to one endpoint: POST /zones/{id_or_name}/rrsets/{rr_name}/{rr_type}/actions/change_ttl. After the deadline, its request must carry either a numeric ttl or an explicit null to adopt the zone default. An omitted value will fail. Hetzner's changelog is unusually clear on both the deadline and scope.

That distinction is the whole incident-prevention exercise. Here is the decision table to use before treating the change as a wildcard-renewal emergency.

The table makes the correct response smaller and more rigorous: find the one call, prove whether it exists, fix it only where it does, and rehearse renewal before the deadline.

Why a TTL endpoint can still reach a wildcard certificate​

Wildcard certificates are why DNS automation has a larger blast radius than a routine A-record update. Let’s Encrypt requires DNS-01 for wildcard issuance. The ACME client receives a challenge, writes a token to _acme-challenge.<domain>, waits for authoritative DNS to serve it, and then asks the CA to validate it. The same process is used at renewal time, not just initial issuance. Let’s Encrypt’s challenge guide also notes two details that matter in real platform fleets: wildcard and non-wildcard authorizations can need multiple TXT values at the same name, and stale values should be cleaned up so the response does not grow without bound.

Many standard integrations only need to add and remove those TXT values. They do not normally need to alter the RRSet TTL. But teams often have a shared DNS abstraction underneath their ACME tool: a Terraform provider, a homegrown reconciler, an ExternalDNS-adjacent library, or a helper that “ensures the record has our preferred TTL” before it appends a TXT value. That abstraction can call change_ttl even when the operator thinks they only configured an ACME solver.

The failure pattern is awkward:

1. A release or provider update preserves an empty TTL field because Hetzner has accepted it.
2. A later reconcile calls change_ttl and receives a validation error after September 30.
3. The ACME client never presents its TXT value, or exits before validation.
4. The running certificate continues to work until its normal renewal window or, in the worst case, expiry.

The risk is therefore not “every Hetzner DNS-01 renewal stops.” It is “a narrow, untested call in the renewal path can fail late enough to be expensive.” That is a much better problem statement because it produces a testable control.

Find the request, not just the package name​

Start with an inventory of the components that can write DNS. In Kubernetes, that includes cert-manager’s DNS01 webhook, any external DNS controller sharing credentials, jobs or CronJobs that run lego, Traefik or Caddy’s ACME provider, and custom controllers. cert-manager supports out-of-tree DNS01 webhooks, including Hetzner’s webhook, so the solver code is a real dependency rather than a built-in cert-manager implementation. The cert-manager DNS01 documentation lists that model and its provider boundary.

Then search the source and rendered configuration for the endpoint, rather than assuming a version is safe:

rg -n 'change_ttl|actions/change_ttl' . \
  --glob '!node_modules/**' --glob '!vendor/**'
 
kubectl get deploy,job,cronjob -A -o yaml | \
  rg -n 'cert-manager-webhook-hetzner|lego|traefik|caddy|acme|dns'
 
kubectl get clusterissuer,issuer -A -o yaml

For an image you did not build, inspect the release notes and source at the image digest you run, or capture a staging renewal through an HTTP proxy. A chart label such as cert-manager-webhook-hetzner is evidence of where to look, not proof of which Cloud API action it sends.

Do the same outside Kubernetes. A Certbot manual authentication hook is ordinary executable code; check the files named by --manual-auth-hook and the renewal configuration. A lego deployment is worth checking for its provider and version, but its documented Hetzner provider already exposes a concrete HETZNER_TTL value with a default of 120 seconds. That is useful evidence that a client has made a deliberate DNS-01 TTL choice, not evidence by itself that it invokes Hetzner’s separate change_ttl action. lego’s Hetzner provider reference documents the setting and its scope.

Finally, include the migration boundary in the inventory. Hetzner made DNS generally available in the Cloud API in November 2025 and stopped creating zones in the older DNS console. The legacy API and the current Cloud API use different endpoints and credentials. A failed migration can look superficially like a TTL failure, especially when an ACME plugin is still pointed at dns.hetzner.com; the symptoms and remediation are different.

Patch the contract explicitly​

If the audit finds a change_ttl request, make the intent explicit. Use a number when the automation owns the RRSet TTL; use null when it should inherit the zone’s default. These are the two supported request shapes after September 30:

{ "ttl": 60 }

{ "ttl": null }

The API action is not a generic “update record” endpoint. It specifically changes the TTL for one RRSet, so do not copy this payload into the calls that create, append, replace, or remove ACME TXT values. Doing so is cargo-cult compatibility and can introduce a new integration bug.

For challenge records, a modest, explicit TTL such as 60 or 120 seconds can be sensible when your automation owns that record set: it bounds recursive-cache staleness without pretending that DNS propagation is instant. It is not a universal magic number. Let’s Encrypt points out that anycast DNS and provider propagation behavior make global visibility hard to predict; a client must still wait long enough and check the right authoritative path. A zone-default null is equally valid when the DNS policy deliberately centralizes TTLs.

One operational wrinkle is shared RRSet state. A wildcard order and an apex order can both place different TXT values under _acme-challenge.example.com. Keep all values during validation and remove only the value your run created. Treat the TTL as a property of the shared RRSet, not an attribute of one token. That avoids a “fix” which changes the TTL or overwrites values while another renewal is in flight.

Rehearse renewal as a failure, not an issuance demo​

A production certificate being valid today does not prove that its next renewal will work. The useful pre-deadline test is a controlled staging order that takes the same code, credentials, zone, and DNS01 route as production. Do it in this order:

1. Record the deployed image digest or binary version, the Cloud DNS zone, and the exact solver configuration.
2. Use the ACME staging directory and request a disposable name or a non-serving test wildcard under a zone you control.
3. Enable HTTP request logging only long enough to verify that the intended DNS actions succeed. Redact bearer tokens and challenge values from retained logs.
4. Confirm the TXT answer from an authoritative nameserver, then confirm that the ACME order reaches a valid state.
5. Exercise cleanup and check that other simultaneous TXT values remain intact.
6. Save the evidence with the dependency version and test date, then repeat the test after upgrades to the solver, chart, or DNS client.

For cert-manager, inspect CertificateRequest, Order, and Challenge resources as well as controller logs. A successful Certificate object is the final signal; an error on the provider’s HTTP action is the earliest one. If you delegate _acme-challenge with a CNAME, include that in the rehearsal. cert-manager defaults to not following challenge CNAMEs, and its documentation calls out that enabling cnameStrategy: Follow can interact unexpectedly with a wildcard CNAME unless an explicit _acme-challenge record exists. That configuration nuance can otherwise masquerade as a DNS provider regression.

Pair the rehearsal with expiry observability. Alert well before certificates approach expiration, and alert specifically on renewal failures or ACME Challenge failures. A dependency deadline should not be discovered by a browser warning at the edge of a tenant domain.

Make small API deadlines part of platform operations​

The durable lesson is not that a 60-second TTL prevents all ACME trouble. It is that a provider’s deprecation notice is an input to the same change-management loop as a Kubernetes version or an image CVE: identify the exact API surface, find every compiled caller, prove behavior in staging, and make the outcome observable.

For a self-hosted PaaS, certificate automation is shared platform machinery. A Bex-style git-push workflow can keep tenant services on machines the operator owns, but it still depends on a deliberately tested DNS and ACME path for HTTPS. Centralizing that path makes the audit repeatable; treating a one-line API contract as “probably handled by the plugin” does not.

Before September 30, the actionable outcome is refreshingly concrete: search for change_ttl; if it is absent from your renewal path, document the evidence and stage a normal renewal anyway. If it is present, send ttl or null, run the staging order, and retain the result as the compatibility test for the next dependency update.

Sources​

• Hetzner Cloud changelog: omitted RRSet TTL deprecation
• Let’s Encrypt: challenge types and DNS-01 behavior
• cert-manager: DNS01 configuration and external webhooks
• lego: Hetzner DNS provider settings