step-ca vs Let's Encrypt: What an Internal-Only ACME CA Buys Your Fleet's Service-to-Service TLS

Every certificate Let's Encrypt issues gets written into a permanent, publicly queryable record. That is not a leak or an oversight — it is the design. Let's Encrypt submits every certificate to Certificate Transparency (CT) logs, including its own Sycamore and Willow logs, and its FAQ warns matter-of-factly that "automated CT crawling bots" will discover your domains shortly after issuance. For your public marketing site, that transparency is a feature. For etcd-0.infra.example.com, vault.internal.example.com, or the mTLS endpoints between your platform's own services, it is free reconnaissance handed to anyone who can type a domain into crt.sh.

There is a second, harder wall: Let's Encrypt only issues certificates for publicly resolvable domain names. Your cluster-internal hostnames, air-gapped segments, and private-network services structurally cannot get a public certificate at all. So a team running its own fleet faces a fork — either contort internal services into the public DNS namespace (and publish their names forever), or run an internal certificate authority. Smallstep's step-ca, an Apache-2.0 ACME-compatible CA you host yourself, is the strongest open-source answer to that fork. Here is what the two options actually look like side by side:

The rest of this post unpacks that table for a concrete environment: a Cluster API-managed fleet running a self-hosted PaaS, where cert-manager already handles public edge TLS and the question is what to do about everything behind it — and what running a second, smaller CA honestly costs you.

Let's Encrypt Is Public-Only by Design, Not by Accident​

Let's Encrypt's constraints are baked into what a publicly trusted CA is allowed to be. Domain validation requires proving control of a name the CA can observe — via an HTTP challenge it can fetch or a DNS record it can query. A name that only resolves on your private network fails that test by definition. Let's Encrypt's own documentation on certificates for local development is blunt about the boundary: for private names, it recommends generating your own certificates or running your own certificate authority, because names like localhost aren't "rooted in a top level domain" that anyone uniquely owns.

Certificate Transparency is the same story. CT exists so that misissuance by any of the hundreds of publicly trusted CAs can be detected by anyone. Let's Encrypt states plainly that it "submits all certificates we issue to CT logs," and lifetimes are equally non-negotiable: 90 days for standard certificates, six days for the short-lived profile, and — quoting the FAQ — "there is no way to adjust these lifetimes, there are no exceptions."

The trouble starts with the common workaround. Teams that want automated certs for internal services often bend them into the public namespace: real subdomains like etcd-0.infra.example.com validated via dns-01, or a public wildcard stretched across internal ingress. The wildcard hides individual names but shares one private key across every service that terminates with it. The per-subdomain route is worse in a different way: each issuance writes the hostname into CT logs that are public, append-only, and crawled by exactly the enumeration tooling attackers use for subdomain discovery. Your internal topology — which databases exist, how many etcd nodes you run, what your admin panels are called — becomes a permanent public index. An internal CA makes the whole contortion unnecessary.

What step-ca Actually Is​

step-ca is Smallstep's open-source certificate authority: Apache-2.0 licensed, roughly 8.7k GitHub stars, and built around the same ACME protocol Let's Encrypt popularized — which means every ACME client you already use (certbot, acme.sh, Caddy, Traefik, cert-manager, lego) can talk to it by swapping one directory URL.

Standing one up is short:

# Initialize a root + intermediate CA
step ca init
 
# Add an ACME provisioner
step ca provisioner add acme --type ACME

Clients then point at your directory URL — https://ca.internal:9000/acme/acme/directory — instead of Let's Encrypt's. For validation, step-ca supports the three standard ACME challenges (http-01, dns-01, tls-alpn-01) plus device-attest-01, which binds certificates to hardware identity via TPM, Secure Enclave, or YubiKey. Because the CA sits inside your network, http-01 works against names that never resolve publicly.

ACME is only one provisioner among several. step-ca also authorizes issuance via JWK tokens, OAuth OIDC identity tokens, cloud instance identity documents (AWS, GCP, Azure), SCEP, and X5C certificate chains — useful when the thing requesting a certificate is a machine, a CI job, or a Kubernetes controller rather than a web server answering challenges.

Two defaults deserve attention because they encode a philosophy. First, the default TLS certificate lifetime is 24 hours — not 90 days — and it is configurable per provisioner:

"claims": {
  "minTLSCertDuration": "5m",
  "maxTLSCertDuration": "24h",
  "defaultTLSCertDuration": "24h"
}

Second, step-ca leans on passive revocation: rather than maintaining CRLs or OCSP responders, you revoke a certificate by blocking its renewal and letting the short lifetime expire it. A stolen key that dies within 24 hours is a much smaller prize than one valid for three months. Short lifetimes are the mechanism that makes revocation-by-expiry credible — and, as we will see, they are also the mechanism that makes your CA's uptime matter.

Wiring step-ca into a Cluster API Fleet​

On a Kubernetes fleet the integration point is cert-manager, which most platforms already run for public edge certificates. There are two clean paths from cert-manager to step-ca.

Path 1: cert-manager's ACME issuer, pointed at step-ca. cert-manager explicitly supports private ACME servers — any server that follows the ACME spec — via a custom server URL, and since v1.11 you can hand it your internal root with the caBundle field:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: step-ca-acme
spec:
  acme:
    server: https://ca.internal:9000/acme/acme/directory
    caBundle: <base64 PEM of your internal root>
    privateKeySecretRef:
      name: step-ca-acme-account
    solvers:
      - http01:
          ingress:
            ingressClassName: internal

The caveat: cert-manager's ACME issuer only implements http-01 and dns-01 solvers, and challenge plumbing designed for public CAs is overhead you may not want for pod-to-pod certificates.

Path 2: step-issuer, Smallstep's native cert-manager integration. step-issuer (also Apache-2.0) is a controller that watches cert-manager CertificateRequest resources and forwards them straight to step-ca using a JWK provisioner — no ACME challenges at all. It adds two CRDs, StepIssuer and StepClusterIssuer:

apiVersion: certmanager.step.sm/v1beta1
kind: StepClusterIssuer
metadata:
  name: step-ca-internal
spec:
  url: https://ca.internal:9000
  caBundle: <base64 PEM of your internal root>
  provisioner:
    name: k8s-jwk
    kid: <provisioner key id>
    passwordRef:
      name: step-ca-provisioner-password
      key: password

From there, internal mTLS is just a cert-manager Certificate — here, a serving-plus-client cert for one platform component talking to another:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: build-queue-mtls
spec:
  secretName: build-queue-mtls-tls
  duration: 24h
  renewBefore: 8h
  usages: ["server auth", "client auth"]
  dnsNames:
    - build-queue.platform.svc.cluster.local
  issuerRef:
    group: certmanager.step.sm
    kind: StepClusterIssuer
    name: step-ca-internal

What should be issued this way on a self-hosted PaaS? Everything with an internal-only name: admission and conversion webhook servers, the internal container registry, the build queue, metrics and log shippers, and the platform's own service-to-service mTLS mesh. The public rule of thumb is simple — if the name is meant to resolve from the internet, Let's Encrypt; if it is not, step-ca.

There is one piece of plumbing that is specifically a Cluster API concern: root distribution. A private CA is only useful if every node trusts it, and on a Cluster API fleet nodes are cattle — machines get replaced by rolling a MachineDeployment, and any trust store you configured by hand dies with the old machine. The internal root has to live in the bootstrap path, for example in a KubeadmConfigTemplate:

spec:
  template:
    spec:
      files:
        - path: /usr/local/share/ca-certificates/internal-root.crt
          content: <PEM of your internal root>
      preKubeadmCommands:
        - update-ca-certificates

Now every node — including ones provisioned three months from now during a scale-up you didn't watch — trusts the internal CA from first boot. Bake it into the machine template once and machine replacement stops being a trust event.

The Honest Cost: You Now Run a CA​

Everything above is the "buys" column. The price is that a service which used to be someone else's globally redundant infrastructure is now a workload on your SLO dashboard. Concretely, the new duties are:

• Availability. With 24-hour certificates, renewal is constant. The standard practice is renewing at two-thirds of lifetime — for a 24h cert, at hour 16 — which leaves an 8-hour buffer between the first renewal attempt and expiry. Run the derivation the other way and it becomes an ops requirement: a step-ca outage longer than about 8 hours starts expiring live certificates fleet-wide. Let's Encrypt outages were never your pager's problem; this is.
• Root and intermediate rotation. Public CAs rotate their hierarchies on their own schedule and browsers ship the updates. Your internal root's expiry, its re-issuance, and the awkward overlap window where both roots must be trusted are now items on your calendar — and the distribution mechanism is the same Cluster API bootstrap path above.
• Key custody. The intermediate signing key is the crown jewel of your internal trust domain. Backups, access control, and ideally hardware or KMS backing are your responsibility.
• Expiry monitoring. Passive revocation means nothing alerts you when renewal quietly breaks — the certificate just runs out. Alert on time-to-expiry from metrics, not on renewal-job exit codes.

The certificate-lifetime knob is where these costs trade against the benefits. Stretch lifetimes to 30 days and a CA outage becomes a non-event — but a compromised key now lives for a month, and passive revocation loses its teeth. Keep 24 hours and revocation-by-expiry is genuinely fast — but your CA needs real redundancy. There is no free setting; pick the failure mode you can live with, per provisioner, and write the choice down.

Where the Line Sits​

The end state is not step-ca instead of Let's Encrypt — it is a clean split along exactly the boundary Let's Encrypt itself draws. Public, internet-resolvable names stay on the public CA: free, automated, and CT-logged in a namespace that is already public. Internal-only names move to step-ca: automated by the same ACME machinery and cert-manager resources, with 24-hour lifetimes, and with your topology kept out of a permanently queryable global record. One ClusterIssuer for the edge, one StepClusterIssuer for everything behind it, and an internal root baked into your machine bootstrap so trust survives node replacement.

The pattern generalizes: a public CA's scope ends exactly where your network begins, and pretending otherwise either blocks automation or publishes your architecture. As fleets get more automated — including ones where AI agents provision services and request certificates without a human in the loop — the platforms that work best are the ones where internal identity issuance is as boring and API-driven as public issuance became a decade ago. An internal ACME CA is how you get there without inventing anything new.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Bex uses cert-manager for tenant-facing TLS out of the box, which is exactly the setup this post's split applies to: public certs for your apps' domains, an internal issuer for the platform's own components. Star the repo on GitHub or deploy your first app today.

Sources​

• Let's Encrypt FAQ — certificate lifetimes (90-day and 6-day, "no exceptions"), CT submission, and CT crawling bots discovering domains.
• Let's Encrypt: Certificate Transparency Logs — all certificates submitted to CT; Sycamore and Willow logs.
• Let's Encrypt: Certificates for localhost — why public CAs can't issue for private names, and what to do instead.
• smallstep/certificates (step-ca) on GitHub — Apache-2.0 license, provisioner types, ACME challenge support.
• Smallstep: Run your own private ACME server — internal hostnames, custom validity, ACME provisioner setup.
• Smallstep docs: ACME basics — challenge types including device-attest-01.
• Smallstep docs: basic CA operations — 24-hour default lifetime, claims configuration, renewal at two-thirds lifetime.
• cert-manager docs: ACME issuer — private ACME server support, caBundle (v1.11+), HTTP01/DNS01 solvers.
• smallstep/step-issuer on GitHub — StepIssuer/StepClusterIssuer CRDs, JWK-based cert-manager integration.