Skip to main content

Why Talos Linux's Own Creator Didn't Build Omni on Cluster API

9 min readDora NodaDora Noda
Share
On this page

Sidero Labs builds Talos Linux, the immutable, API-only OS a Cluster API fleet can run on every node. Sidero also maintains the Cluster API bootstrap and control-plane providers (CABPT and CACPPT) that make Talos a first-class citizen inside Cluster API — the same providers a CAPH-based Hetzner fleet reaches for when it wants Talos instead of Ubuntu-plus-kubeadm. And yet, when Sidero built Omni — its own multi-cluster Kubernetes management SaaS — it didn't build it on Cluster API. It said so, in public, in a blog post titled plainly "Why Omni doesn't use Cluster API."

That's worth sitting with. The company closest to Cluster API's Talos integration, with more hands-on exposure to CAPI's failure modes than almost anyone outside the Cluster API project itself, looked at its own product and walked away. bex runs the opposite bet: Cluster API, with CAPH on Hetzner and CAPD for local development, underneath every tenant deploy. So which one of us is wrong?

Neither, as it turns out — but only because Sidero's objections are specific to what Omni had to be, not to what Cluster API is. Below is a point-by-point read of the six objections from Sidero's post, checked against what a Cluster-API-first fleet like bex's actually looks like in practice.

The verdict, if you're skimming

Sidero's objectionLands on bex?
1. Dedicated management-plane taxNo — bex's one management cluster is amortized across every tenant on the fleet
2. Steep primitives-and-provider learning curveNo — tenants never see CAPI; the cost is paid once by bex's platform team
3. Rolling-replace-only model breaks stateful/edge nodesPartial — fine for stateless workers, real cost on GPU/sandbox node pools with large local caches
4. No mixing providers within one clusterNo — bex is single-provider (Hetzner cloud/bare metal), never needed a hybrid cluster
5. PXE-based provisioning doesn't fit a check-in/edge flowNo — bex provisions in a datacenter via Hetzner's API, not disconnected edge hardware
6. Talos's native in-place upgrade API outguns CAPI's rolling-update primitiveYes — a real capability bex gives up for CAPI's uniform declarative model

Two of the six land as genuine costs bex accepts and mitigates, not costs it evades by luck. The other four are specific to problems Omni's edge-and-homelab target market has that a Hetzner-fleet PaaS doesn't. Here's why, objection by objection.

Objection 1: the management-plane tax

Sidero's complaint: Cluster API needs a Kubernetes cluster just to manage Kubernetes clusters. For someone who wants to run one or two clusters on hardware they already own, standing up and operating an entire second control plane to get there is a bad trade — expensive in machines, expensive in operational attention, for a payoff that's mostly automation they didn't strictly need at that scale.

That math changes entirely once the management cluster isn't serving one customer's one or two clusters — it's serving every tenant on the fleet. bex's Cluster API management cluster provisions and reconciles Machines for the shared pool every tenant's app eventually lands on. The fixed cost of running that control plane gets divided across however many tenants are on the platform, not multiplied by each one standing up their own. Sidero even names this distinction directly in its own post: CAPI is "still a good choice for particular use cases — large-scale provisioning of many clusters." That's a description of bex's actual position, not Omni's.

Objection 2: the primitives-and-provider learning curve

Sidero's second complaint is aimed at a specific user: someone who is new to Kubernetes, wants Omni's simplicity precisely because they don't want to learn Cluster, MachineDeployment, KubeadmControlPlane, and a stack of interlocking custom resources on top of the pods and deployments they're already trying to understand.

bex's tenants are never that user, because they never touch Cluster API at all. A tenant pushes a git repo and gets a running HTTPS service; the CAPI object graph underneath is bex's own platform team's problem, not something a developer deploying an app has to reason about. The learning curve is real, but it's paid once, by the team that already has to understand Kubernetes to run the platform in the first place — not repeatedly, by every user Omni was trying to protect from exactly this.

Objection 3: rolling-replace-only upgrades vs. stateful and edge nodes

This is where the verdict gets more interesting than "doesn't apply." Cluster API's default upgrade model treats every Machine as disposable: cut a new one, join it, drain and delete the old one. Sidero's point is that this breaks down hard for a homelab node with no spare hardware to rotate in, or for an edge box carrying a large local Ceph volume that a fresh replacement Machine doesn't have.

For bex's core fleet — the general worker pool that runs tenant application containers — this mostly isn't a problem. Those nodes are meant to be interchangeable; tenant persistent state lives off-node by design (bex doesn't run managed databases, so tenant data already lives somewhere that survives a worker being replaced). Rolling replacement is close to free there.

It stops being free the moment a node pool exists specifically because it's carrying something expensive to reload: a GPU node running an AI-agent sandbox with tens of gigabytes of model weights cached locally, for instance. Replacing that Machine on a Cluster API upgrade means re-downloading that cache from scratch before the node is useful again — a real, measurable cost every time a rolling upgrade touches that pool, in a way it never touches the stateless general fleet. This is the one objection where the honest answer is "it depends on which node pool you're asking about," not a clean yes or no.

Objection 4: no mixing providers within one cluster

Sidero built Omni to support genuinely hybrid clusters — Sidero's own example is a control plane on Azure with worker nodes on Equinix Metal, connected over Omni's KubeSpan networking. Cluster API's provider model doesn't support that: a cluster's infrastructure provider is one provider, full stop, and nothing in CAPI lets an operator mix providers for different node roles within the same cluster.

bex never asked that question in the first place. Every bex-managed cluster is single-provider: CAPH against Hetzner in production, CAPD against local Docker for development. There's no cross-cloud control-plane-here-workers-there topology to support, so a limitation that's a hard blocker for Omni's product requirements is simply a feature bex never needed.

Objection 5: PXE provisioning vs. a check-in flow

Omni's edge story assumes hardware that isn't reachable the way a datacenter machine is: a box gets imaged once, boots, opens a WireGuard tunnel back to Omni, and announces itself as an unallocated machine — a "check-in" flow, because PXE network-boot provisioning assumes a network Omni's edge deployments can't assume they have.

bex's fleet lives in a Hetzner datacenter, provisioned through Hetzner's own Cloud and Robot APIs. CAPH talks to that API directly to bring machines up; there's no PXE-vs-check-in tension to resolve because there's no disconnected edge hardware in the picture at all. This objection is entirely a function of Omni's edge target market, and it has no analog in a fleet that's always reachable over a normal datacenter network.

Objection 6: Talos's own upgrade API vs. CAPI's rolling-update primitive

This is the second place the honest answer isn't "no." Talos ships its own upgrade API through talosctl — a direct, lower-level path for upgrading both Talos itself and the Kubernetes version running on top of it, without necessarily replacing the Machine at all. Sidero built Omni to call that API directly. Cluster API's Talos bootstrap and control-plane providers (CABPT and CACPPT) do wrap that same upgrade API, but they drive it through CAPI's own MachineDeployment rolling-update controller rather than exposing Talos's native path directly.

That's a real trade bex makes, not a false alarm. Going through CAPI buys bex one uniform declarative upgrade model across its entire fleet — the same reconciliation loop handles a Talos upgrade the same way it handles everything else Cluster API manages. What it gives up is Omni's more direct route to Talos's own upgrade primitive, which can be faster and less disruptive precisely because it isn't routed through a generic rolling-replace abstraction built to work the same way regardless of node OS. Sidero built Omni to avoid paying this cost; bex pays it deliberately, in exchange for one consistent operational model instead of a bespoke one.

Why the same six objections point two different ways

Sidero's own post names the actual dividing line: Cluster API fits "large-scale provisioning of many clusters, without the need for hybrid clusters" and fits poorly for a solo operator who wants one or two clusters running on hardware with no spare capacity to rotate. Omni was built for the second profile. bex — one Cluster API management plane, one Hetzner provider, a shared fleet serving every tenant that pushes a git repo — is built for the first.

That's why four of Sidero's six objections (the management-plane tax, the learning curve, provider mixing, and PXE-vs-check-in) simply don't describe bex's architecture — they describe constraints specific to a product aimed at edge hardware and Kubernetes newcomers running their own gear. The other two (stateful-node upgrades and Talos's native upgrade API) are real, and bex accepts them rather than argues them away: a GPU-heavy sandbox pool pays a real replacement cost on upgrade, and every Talos upgrade goes through CAPI's rolling-update model instead of talosctl's more direct path. Six objections were enough to send Sidero away from Cluster API for Omni. The same six objections, read against a different architecture, are a reason to stay.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex