The Tenant Domain That Didn't Renew: cert-manager vs Gardener vs certctl for Per-Tenant TLS

Every self-hosted platform meets this incident eventually: a tenant's custom domain stops renewing its certificate at 2am, the status page lights up, and the postmortem reveals the renewal pipeline was designed for ten domains and is now serving four hundred. TLS automation that works fine for a platform's own dashboard becomes a different engineering problem the moment every tenant brings their own domain.

Two numbers frame the entire problem. Let's Encrypt — the default public CA behind nearly every automated setup — issues at most 50 certificates per registered domain per week, with only 5 duplicate certificates per week and 300 new orders per ACME account per 3 hours. And wildcard certificates can only be issued via DNS-01 challenges — never HTTP-01. Every architecture decision below is downstream of those two constraints.

The short version: if you run one Kubernetes-native fleet, default to cert-manager with DNS-01 and a shared ACME account, and design around the 50-per-week ceiling from day one. If your fleet is already Gardener-managed, use Gardener cert-management — its annotation-driven model and shoot-cert-service are the cheapest path to multi-cluster TLS. Reach for certctl when private keys must never leave the host, when you juggle many CAs, or when targets aren't all Kubernetes. Details and deviation conditions below; the full head-to-head table closes the piece.

The actual problem: N tenants, N domains, one expiry cliff​

Strip away the tooling and per-tenant TLS is three coupled problems. First, challenge type: HTTP-01 proves control by serving a token over port 80 — simple, but it cannot issue wildcards and breaks behind restrictive ingress. DNS-01 proves control via a TXT record, which unlocks wildcards but needs API credentials for every tenant's DNS zone — credentials your platform must store, scope, and rotate.

Second, secret distribution: the issued certificate has to land where traffic terminates — the right namespace, the right cluster, the right edge proxy — before expiry. Third, renewal monitoring: something must watch hundreds of independent expiry dates and scream before, not after, the cliff.

Do the scaling math once and the cliff becomes concrete. Take 200 tenants, each with one custom domain, all onboarding in a launch week. At one certificate per domain against Let's Encrypt's 50-per-registered-domain-per-week ceiling, tenants 51 through 200 simply fail issuance until the window rolls over — unless subdomains share a wildcard (one DNS-01 issuance covering all of them) or issuance is spread across ACME accounts and CAs. The Cozystack platform hit exactly this wall: every published hostname minted its own per-host ACME certificate via ingress-shim, which works until roughly 50 tenant endpoints and then collides with the rate limit; their fix was issuing a single per-apex wildcard in DNS-01 mode. That is the representative workload for everything that follows: not one cert, but issuance throughput under a hard external quota.

cert-manager: the Kubernetes-native default​

cert-manager is the de facto answer for a reason: it models certificates as Kubernetes resources. An operator annotates an Ingress (the ingress-shim path) or declares a Certificate CR, references an Issuer or ClusterIssuer, and controllers drive ACME orders, store the result in a Secret, and renew it on a schedule. DNS-01 is covered by webhook providers for most DNS APIs; trust distribution beyond a single namespace is handled by its companion trust-manager, which fans CA bundles out as ConfigMaps (recent releases added a restricted mode scoping caches to specific namespaces, and migrated the Bundle API to ClusterBundle).

What it costs at tenant scale:

• CRD sprawl is the scaling surface. Hundreds of tenants mean hundreds of Certificate objects, Orders, and Challenges churning through etcd and the ACME server. It works — this is the most battle-tested path in the ecosystem — but your monitoring must cover the controller's work queue, not just the certs.
• One ACME account is a shared fate. All Certificates behind one issuer share the account's 300-orders-per-3-hours budget and the domain's 50-per-week ceiling. cert-manager has supported reusing one ACME account across clusters (by copying the account key secret) since v1.0, which concentrates rate-limit exceptions — but also concentrates blast radius. A renewal thundering herd after an outage can exhaust the order budget and delay every tenant at once; stagger renewal windows.
• DNS-01 credentials per tenant zone. For custom domains, someone holds DNS API tokens for zones you don't own. Scope them per-tenant, rotate them, and treat that secret inventory as part of the cost — it is the least automatable part of the stack.
• Renewal failure modes are silent by default. A failed Challenge re-queues with backoff and logs; nothing pages unless you alert on cert-manager metrics (certmanager_certificate_expiration_timestamp_seconds, ready-condition flaps). The 2am incident above is what happens when nobody wires those metrics up.

Default here means: DNS-01 ClusterIssuer, one shared ACME account per trust boundary, wildcard-per-apex where tenants live under your domain, per-domain certs only for true custom apexes, and alerts on expiry horizon from day one.

Gardener cert-management: certificates as shoot metadata​

Gardener's cert-management takes a different posture: TLS is a property of the managed fleet, not of each workload. Annotate an Ingress with cert.gardener.cloud/purpose: managed and the cert-controller-manager handles issuance for the discovered domains; on Gardener shoot clusters the shoot-cert-service extension (auto-enableable for every shoot via ControllerRegistration) plus Gardener's DNSRecord integration (dns.gardener.cloud/class: garden) closes the loop from DNS record to served certificate, including custom domains and custom shoot-level issuers declared through CertConfig.

What it costs at tenant scale:

• Cheapest if you're already on Gardener; heaviest if you're not. Inside a Gardener landscape, per-shoot TLS is nearly free operationally — one annotation, fleet-wide issuer policy, DNS records reconciled by the same control plane. Outside Gardener, adopting the whole landscape runtime to get certificate automation is disproportionate; cert-management assumes Gardener's DNS and shoot machinery.
• Multi-cluster is the native shape. The shoot-cert-service exists precisely so N clusters share issuance policy without N copies of cert-manager config. A Cluster-API fleet running many workload clusters gets the same benefit only by federating cert-manager (shared ACME account secrets, per-cluster issuers) by hand.
• Issuer secrets are first-class. Custom CAs and per-shoot issuers ride in Kubernetes secrets with explicit domain-include scoping, which maps cleanly onto "tenant A's domains may only use issuer X." That is genuinely nicer than cert-manager's flatter issuer model for multi-tenant boundaries.
• Renewal failure modes live in Gardener's event stream. Debugging means reading cert-controller-manager and DNS controller state rather than cert-manager's familiar Certificate/Order objects — a smaller community and fewer Stack Overflow answers when it breaks at 2am.

If your control plane is Gardener, this is the default and the comparison is over. If it is Cluster API on bare metal, treat Gardener's model as the design to admire, not the dependency to adopt.

certctl: the heterogeneous-fleet control plane​

certctl answers a different question: what if the TLS estate isn't all Kubernetes? It is a self-hosted control plane (server plus per-host agents) covering the full lifecycle — issuance, renewal, deployment — across twelve native CA connectors (Let's Encrypt and any ACME provider, step-ca, Vault PKI, EJBCA, AWS ACM PCA, Google CAS, DigiCert, Sectigo, GlobalSign, Entrust, a built-in local CA with sub-CA mode, plus an OpenSSL/shell-script adapter for anything custom) and fourteen deployment targets (servers, appliances via a proxy-agent pattern, Kubernetes Secrets in preview, and more).

What it costs at tenant scale:

• A second control plane to run. The server, its database, and an agent on every target host are new infrastructure with their own upgrades, backups, and monitoring — justified when targets are heterogeneous (VMs, appliances, mixed clusters), overhead when everything already terminates in one ingress.
• The security posture is the payoff. In agent mode (the default), private keys are generated on the host and never touch the control plane; server-side key generation exists only behind explicit demo-mode flags. For regulated tenants or customer-managed keys, "keys never leave the host" is a property neither cert-manager (Secrets in etcd) nor Gardener's model gives you without extra work.
• CA agility is built in. Multi-CA failover — Let's Encrypt for the bulk, a commercial CA for tenants with compliance requirements, a local CA for internal domains — is a connector configuration, not a second automation stack. Against a single CA's rate limits, that is also a scaling lever: spread issuance across CAs instead of queuing behind one 50-per-week ceiling.
• Kubernetes is a preview, not the home turf. If every tenant terminates at your ingress controller, certctl's agent model adds moving parts where cert-manager's in-cluster controllers need none. Its embedded ACME server is a useful bridge here — point cert-manager or lego at it and keep existing flows while centralizing CA policy.

Reach for certctl when the estate spans host types, when key custody is a requirement rather than a preference, or when one CA's quota is the bottleneck and you need issuance spread across several.

Head-to-head: what each one actually costs​

The default, with conditions. On a Kubernetes-native Cluster-API fleet where tenants terminate at your ingress: cert-manager, DNS-01, shared ACME account per trust boundary, wildcard-per-apex under your domain, per-domain certs only for true custom apexes, expiry-horizon alerts before the first tenant onboards.

Deviate when the premise changes: already on Gardener → cert-management; keys must not leave hosts, targets aren't all Kubernetes, or one CA's quota binds → certctl (possibly fronted by cert-manager via its embedded ACME server during migration).

Whichever you pick, the failure that pages you at 2am is never "which tool" — it is the unmonitored expiry, the unscoped DNS credential, or the fifty-first certificate in a week. Automate issuance, but operate renewal: alert on a 14-day horizon, rehearse CA failover, and re-run the 200-tenant onboarding math against real quotas before launch week runs it for you.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.