Karmada for Cluster API Fleets: When One Control Plane Stops Being Enough

Every self-hosted platform starts the same way: one Cluster API management cluster, one or two workload clusters, and a Git repo full of manifests that kubectl apply handles just fine. Then a tenant asks for workloads in a second region. Then a third cluster appears for GPU sandboxes. Then somebody edits a per-cluster ResourceQuota by hand at 2 a.m. and nobody can say what the fleet-wide limit actually is anymore. The single control plane did not break. It just stopped being the whole answer.

Karmada — the CNCF multi-cloud, multi-cluster orchestrator — is the layer teams reach for at exactly this ceiling: a hub control plane that propagates workloads to member clusters through declarative policies instead of copy-paste scripts. It is a real project with real production users (Trip.com, Vivo, and 20-odd others publicly listed), a v1.17 release that added workload affinity scheduling, and a v1.18 line adding overcommit protection. It is also another distributed system you have to run. Here is the verdict up front, so you can stop reading the moment your situation is clear:

If every row in the left column describes you, Karmada is scope creep. If two or more rows describe the right column, keep reading — the rest of this post is the concrete what, what-it-costs, and what-else before you commit.

Karmada in five minutes (and what it is not)​

First, the guardrail that prevents the most common misunderstanding: Karmada does not provision clusters. It sits above clusters your Cluster API management plane already created. You register each workload cluster as a member, and Karmada's hub — its own apiserver, etcd, scheduler, controller manager, and webhook, plus an agent in every member — decides what runs where. CAPI still owns machine lifecycle; Karmada owns workload placement. They compose; neither replaces the other.

The core object is the PropagationPolicy: a standalone policy that selects resources and declares where they go, separately from the workload manifests themselves. A minimal policy that duplicates a tenant deployment across two Hetzner-region clusters looks like this:

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: web-duplicated
  namespace: tenants
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: web
  placement:
    clusterAffinity:
      clusterNames: [hetzner-fsn1, hetzner-nbg1]
    replicaScheduling:
      replicaSchedulingType: Duplicated

Change Duplicated to a spread or weighted split and you have multi-region rollout without a bespoke fan-out script. A companion OverridePolicy patches per-cluster differences (image registry mirror, region label) on top of the same template, so the base manifests stay identical everywhere.

Version pins matter here, because Karmada is moving fast and roadmap talks are easy to mistake for shipped code. v1.17 (March 2026) headlined workload affinity and anti-affinity scheduling — a WorkloadAffinity stanza in the policy API that co-locates related workloads or spreads competing ones across clusters — plus a Dashboard v0.3.0 release.

v1.18 added overflow cluster affinity for hybrid-cloud spillover and a scheduling overcommit protection mechanism. Items like application priority scheduling, stateful failover, and AI job scheduling enhancements come from the project's KubeCon roadmap talks: direction, not GA. Label them that way in your evaluation.

What it actually buys a CAPI fleet​

Manifest fan-out without the copy script. Today the two-cluster team usually has a shell loop, a Makefile target, or a CI job that applies the same manifests N times and prays the contexts stay in sync. A propagation policy replaces that imperative fan-out with a declared desired state: add a third cluster to clusterAffinity and the hub converges it; remove one and the bindings drain. GitOps tools can live underneath this — Karmada's own docs show Flux CRDs installed as resource templates on the hub — but the placement decision stops being a script somebody owns.

One quota story instead of N unreconciled ones. Per-cluster ResourceQuota objects do not sum, and nobody reconciles them against each other: a tenant can be capped in the cluster with spare capacity while bursting in the full one. Karmada's FederatedResourceQuota (enforcement graduated over the v1.14 line) imposes limits at the hub control plane, so the scheduler checks quota before placing work and skips clusters where the tenant is already exhausted.

Before/after, concretely: before, three clusters each grant a tenant 32 CPU with no shared accounting, so the tenant's real ceiling is anywhere from 32 to 96 depending on scheduler luck; after, the hub enforces one 64-CPU envelope and placement follows the remaining headroom. If your tenants never contend across clusters, this buys you nothing — which is itself a useful filter.

Failover with a real timeline — and an honest data boundary. Karmada's failover model is taint-based and legible. When a member's Ready condition flips, the hub stamps NoSchedule taints (cluster.karmada.io/not-ready, /unreachable) so nothing new lands there; if the cluster stays unhealthy past the eviction grace period (default five minutes, tunable via the controller manager's failover flags), NoExecute eviction reschedules the bindings onto healthy members.

Newer versions gate the aggressive half behind an explicit failover feature and a ClusterTaintPolicy API, so the blast radius of automation is itself declarative.

The boundary that matters most for a PaaS: Karmada moves bindings, not bytes. A stateless web tier fails over on roughly the detection-plus-grace timeline above. A stateful tenant database does not, because its PersistentVolumes live in the dead cluster — failover of state needs storage-level replication, volume snapshots restored cross-cluster, or a Velero-driven restore (Karmada's docs cover Velero integration for backing up hub resources, which is the starting point, not the finish line).

Any "stateful application cluster failover" slide that omits RPO/RTO and the data plane is selling the stateless timeline for a stateful problem. Plan the storage story first; Karmada only decides where the restored pods land.

Scheduling semantics single clusters cannot express. v1.17's workload affinity/anti-affinity lets a policy co-locate an inference server with its cache cluster-side or spread payment microservices so one cluster loss never takes the whole chain. v1.18's overcommit protection stops the scheduler placing work that quota math already doomed. None of this replaces bin-packing inside a cluster — it is placement between clusters, a layer that only exists once you have more than one worth placing across.

AI jobs: scoped, not skipped. The roadmap's "AI jobs scheduling enhancements" deserves a precise boundary because agent-sandbox fleets will ask. Karmada propagates batch/AI jobs across clusters, and Volcano's multi-cluster extension builds on top of Karmada for cross-cluster Volcano Jobs with queue priority. But gang scheduling, fair-share queues, and GPU topology awareness still execute inside member clusters via Volcano or Kueue — Karmada picks the cluster, it does not run the queue.

If your training and inference jobs fit in one cluster's GPU pool, Karmada adds a hop and zero value. Adopt it for AI only when jobs must span clusters and you already run the in-cluster scheduler it leans on.

What it costs (itemized, no hand-waving)​

A second control plane, running forever. The hub is a full stack: etcd, the Karmada apiserver (plus an aggregated apiserver), scheduler, controller manager, webhook, and optional search/descheduler addons — with an agent and scheduler-estimator footprint in every member cluster.

Community HA manifests run etcd at three replicas with pod anti-affinity, apiserver at two, controller manager and scheduler at two behind leader election. On owned hardware that is roughly a dedicated small control-plane cluster before it places a single tenant pod.

It also becomes the new blast radius: if the hub is down, nothing gets placed, even though running workloads keep serving. CAPI already taught you this shape; Karmada doubles it.

Version skew, squared. Every hub upgrade now has a compatibility matrix against every member's Kubernetes version and every agent version. The project's own changelog discipline is good, but the testing burden is yours: a policy that scheduled perfectly on v1.17 semantics needs re-validation when the hub moves to v1.18 while members lag.

Failover is never Karmada-alone — the traffic layer is still yours. Rescheduling pods to a healthy cluster does nothing for users until traffic moves. The published multi-region pattern (one load balancer per region, DNS failover with health checks, short TTLs) is the same on any cloud — the well-known EKS-plus-Route-53 walkthroughs just rent the pieces from AWS.

Self-hosted, that means per-region load balancers in front of each member cluster, external-dns (or equivalent) flipping records on health-check failure, and sub-minute TTLs you have tested under real propagation delay. Budget that DNS and LB layer, its health-check design, and its failover drills as part of the Karmada decision — without it you have pod-level failover and user-facing downtime.

Two layers to debug at 3 a.m. Every incident now asks "is it the member or the hub": a stuck rollout could be a broken member kubelet, a hub scheduler decision, a taint policy misfiring, or skew between them. Karmada's dashboard (v0.3.0 in the v1.17 line) and its operator help, but on-call needs runbooks for both layers from day one.

The alternatives, and when each wins​

The through-line: GitOps tools distribute intent; Karmada schedules. If your question is "get these manifests onto these clusters," ApplicationSets or Fleet is simpler. If your question is "decide continuously which clusters should run this, within quota, surviving a cluster loss," that is the sentence Karmada exists to answer — and the only one that justifies its hub.

The ceiling test​

Karmada is not a better Cluster API; it is the admission that one control plane's blast radius, one region's fate-sharing, and one cluster's quota math no longer describe your fleet. The project has earned a serious look — 23 public production adopters including Trip.com's hybrid multicloud control plane, a published 100-cluster scale test, AWS's own EKS guidance, and a scheduler that keeps gaining genuinely useful semantics. But every benefit above has a threshold underneath it: three clusters, cross-cluster contention, a failover SLA with minutes in it, a team that can run a hub in HA.

Run the table at the top of this post against your fleet once a quarter. The quarter two rows flip to the right column, start the proof of concept: register two CAPI-provisioned members, propagate one stateless tenant service with a Duplicated policy, kill a member, and time the taint-to-serving path end to end — DNS included. That single drill teaches more than any roadmap talk, because it measures the only number that matters: how long your tenants wait while two control planes agree on reality.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Sources​

• Karmada v1.17 release announcement — workload affinity/anti-affinity scheduling, Dashboard v0.3.0 (karmada-io/website blog, March 2026; CHANGELOG-1.17)
• Karmada v1.18 release notes — overflow cluster affinity, scheduling overcommit protection (karmada.io/blog)
• KubeCon project talk — priority scheduling, FederatedResourceQuota enforcement, stateful application failover, AI jobs scheduling, dashboard, operator (video)
• FederatedResourceQuota user guide and enforcement (karmada.io/docs)
• Cluster failover process analysis — taint model, grace periods (karmada.io/docs)
• CNCF review — failover feature gate, ClusterTaintPolicy (karmada-io/community)
• Volcano Global v0.1.0 — multi-cluster AI job scheduling on Karmada (volcano-sh/volcano-global)
• 100-cluster scale test report (karmada.io/blog/2022/10/26/test-report)
• Adopters and Trip.com case study (thenewstack.io)
• AWS guidance for Karmada on EKS (aws.amazon.com)
• Multi-region EKS failover walkthrough with Route 53 (Medium, Jan 2026)
• Karmada vs Open Cluster Management (cncf.io/blog/2022/09/26)
• Velero integration for Karmada backups (karmada.io/docs)