Skip to main content
bex.co
419
Blog

ClusterClass Turns Tenant Cluster #500 Into a One-Line Diff From Cluster #1

8 min readDora NodaDora Noda
Self-Hosting
Kubernetes
Infrastructure
Engineering
Share
On this page

Provision tenant cluster #1 on a Cluster API fleet and you hand-author seven objects: a Cluster, a HetznerCluster, a KubeadmControlPlane, a HetznerMachineTemplate for the control plane, a second HetznerMachineTemplate for workers, a KubeadmConfigTemplate, and a MachineDeployment. Provision tenant cluster #47 and you copy all seven, rename them, and hope you changed every field that needed to differ and none of the ones that didn't. Cluster API has had a fix for this sitting in the project for years — ClusterClass — and if your fleet is still hand-copying manifests per tenant, adopting it is one of the cheapest structural changes available before the copy-paste starts costing you a production incident.

The before/after, concretely​

Here's what "provision tenant cluster #47" looks like under the two models, using cluster-api-provider-hetzner (CAPH) as the concrete provider — the one a Hetzner-backed fleet is already running.

Without ClusterClass, a new tenant cluster is a full copy of the object set: Cluster (with spec.infrastructureRef and spec.controlPlaneRef pointed at brand-new objects), HetznerCluster (network, placement group, control-plane endpoint), KubeadmControlPlane (version, replica count, an embedded KubeadmConfigSpec with the Hetzner cloud-provider config baked in), a HetznerMachineTemplate for the control-plane machine type, a second HetznerMachineTemplate for the worker machine type, a KubeadmConfigTemplate for worker bootstrap, and a MachineDeployment tying version and replica count to the worker template. Every one of those objects is real YAML with real fields — server types, SSH key references, network IDs, kubelet extra args — and every field is duplicated verbatim across tenants except the handful that are actually supposed to differ. Nothing enforces that "supposed to be identical" stays true; a hand-edit to fix a bug on tenant #12 that never gets back-ported to tenant #13 is drift, and it's invisible until it breaks something.

With ClusterClass, those same seven object kinds get authored exactly once, as templates referenced from one ClusterClass resource, plus a variables block declaring the knobs a tenant is actually allowed to set — worker machine type, worker replica count, Kubernetes version — with an OpenAPI schema for each (type, default, validation) and a patches list wiring each variable to the JSON Patch operation that injects it into the right template field. A new tenant's entire manifest collapses to this:

yaml
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: tenant-047
  namespace: tenant-047
spec:
  topology:
    class: hetzner-standard
    version: v1.34.1
    controlPlane:
      replicas: 3
    workers:
      machineDeployments:
        - class: default-worker
          name: md-0
          replicas: 3
          variables:
            overrides:
              - name: workerMachineType
                value: "cpx31"

That's the whole diff between tenant #1 and tenant #47: a name, a namespace, and whichever variable overrides that tenant actually needs. Everything else — the Hetzner network config, the kubeadm bootstrap args, the machine template shape — lives in one ClusterClass object that every tenant cluster references instead of copies. Fix a bug in the shared template once, and every cluster built from that class picks it up on its next reconcile; there's no back-port step to forget.

How ClusterClass is actually put together​

A ClusterClass has three parts. The infrastructure and control-plane templates — for CAPH, a HetznerClusterTemplate and a KubeadmControlPlaneTemplate — describe the shape of the cluster itself. The workers section holds one or more MachineDeploymentClass (and, if you use them, MachinePoolClass) definitions, each referencing its own bootstrap and infrastructure templates; a single MachineDeploymentClass can be reused across many tenant topologies, each supplying different variable overrides for the same class, which is exactly what makes the tenant manifest above so short.

Variables are the customization surface: a name, an OpenAPI schema (string, integer, number, boolean, or a structured object), whether it's required, and a default. This is the deliberate design point — you don't create a second ClusterClass for "the tenant that wants a bigger worker box," you add a variable.

Patches are how a variable's value actually reaches a template field. The workerMachineType override in the tenant manifest above only does anything because the ClusterClass declares a patch like this, wiring the variable to the actual field on the HetznerMachineTemplate the worker MachineDeploymentClass references:

yaml
patches:
  - name: workerMachineType
    definitions:
      - selector:
          apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
          kind: HetznerMachineTemplate
          matchResources:
            machineDeploymentClass:
              names: ["default-worker"]
        jsonPatches:
          - op: replace
            path: /spec/template/spec/type
            valueFrom:
              variable: workerMachineType

Cluster API supports two ways to define patches like this one. Inline patches, as above, are JSON Patch operations (add/remove/replace) against a selector that targets specific templates, with an optional enabledIf Go-template condition for turning a patch on or off per cluster. External patches are implemented via the Runtime SDK as a RuntimeExtension that Cluster API's topology controller calls out to — GeneratePatches, ValidateTopology, and DiscoverVariables are the hooks it exposes. External patches trade a bit of operational complexity (you're now running and versioning a webhook) for handling multiple template API versions in one implementation instead of hardcoding a single apiVersion into an inline patch. For a Hetzner-only fleet with one infrastructure provider, inline patches cover the actual need; external patches start earning their complexity once a platform templates against more than one provider's API version at a time.

What ClusterClass still asks you to get right​

None of this is free, and the gaps are worth knowing before you commit a fleet's provisioning path to it.

It's still labeled experimental. The Cluster API Book lists ClusterClass under "Experimental Feature: ClusterClass (alpha)," gated behind the ClusterTopology feature flag (CLUSTER_TOPOLOGY=true set before clusterctl init), on a management cluster running Kubernetes 1.22 or later. "Alpha" here has meant "stable enough that CNCF case studies like SNCF run national infrastructure on it," not "don't use it" — but it does mean checking the feature-gate default hasn't silently flipped when you upgrade Cluster API, and reading the changelog before every minor bump the way you would for any alpha-labeled surface.

Patches only reach /spec, and array operations only append or prepend — there's no patching an element at an arbitrary index. That's a real constraint if a template's array field (say, a list of extra volume mounts) needs a specific tenant to insert in the middle rather than the ends; the workaround is structuring the template so the position that needs to vary is already at an edge, not discovering the limitation mid-migration.

Variable overrides are per-topology, not automatic. A MachineDeploymentClass shared across tenants only produces different Kubernetes objects because each tenant's Cluster.spec.topology.workers.machineDeployments[].variables.overrides says so — nothing defaults a busy tenant to a bigger box just because its neighbor got one. The design still requires someone to decide, upfront, which fields are exposed as variables at all; a field left out of the ClusterClass's variable schema is a field every tenant is stuck sharing until the class itself is edited.

Cross-namespace ClusterClass references work, but need a guardrail. Cluster.spec.topology.classRef.namespace lets a tenant's Cluster reference a class living in a different namespace, which is exactly what a shared, centrally-maintained class needs to do in a multi-tenant fleet — but without a ValidatingAdmissionPolicy restricting which namespaces can reference which classes, any tenant can point their cluster at any class in the management cluster, including ones they shouldn't be able to touch.

The upgrade path this also buys​

Getting the fleet onto ClusterClass now pays a second, separate dividend later: Cluster API v1.12's chained upgrades — jumping a cluster more than one Kubernetes minor version in a single declared operation instead of stepping through each intermediate version by hand — compute their upgrade plan from information carried on the ClusterClass itself. A fleet that's still hand-copying per-tenant manifests doesn't have a ClusterClass to enhance with that information; it has to retrofit one before chained upgrades are even on the table. Adopting ClusterClass for the boilerplate problem today is also clearing the prerequisite for the upgrade-automation problem tomorrow.

Where this leaves a Hetzner-backed fleet​

CAPH already supports ClusterClass — this isn't a provider gap to wait out. The honest case for adopting it isn't "your fleet is in pain today"; at a handful of tenant clusters, hand-copied manifests are annoying but survivable. It's that the cost of migrating grows with every cluster provisioned the old way, and the failure mode it's insuring against — two clusters that were supposed to be identical quietly drifting apart until one of them breaks in a way the other doesn't reproduce — is exactly the kind of bug that's cheap to prevent structurally and expensive to debug after the fact. "Provision cluster #500 is a one-line diff from cluster #1" is a property you get by building the fleet on ClusterClass before cluster #500 exists, not one you retrofit painlessly once it does.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Sources​

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex
Back to all posts