Colmena vs Cluster API: Nix Fleet Management Without Kubernetes — What It Gets Right and Where It Stops

Every machine in your fleet has two problems: how it got its configuration, and how you know it still matches. Kubernetes answers both with a control plane — an API server holding desired state, controllers reconciling toward it forever. Nix answers both with a build — a cryptographic derivation where the declared configuration and the installed bits are literally the same artifact, so drift is not something you detect but something that cannot exist.

Here is the verdict up front: if your fleet is a set of long-lived machines whose job is to be configured machines — VPN gateways, build farms, monitoring boxes, the control-plane nodes themselves — Colmena gives you fleet-wide declarative management with no control plane, no agent, and no state database, and there is nothing in the Kubernetes world that does it that simply.

If your fleet exists to run tenant containers that arrive, scale, die, and move — Colmena cannot provision a machine, schedule a workload, or discover a service, and bolting those on by hand rebuilds a worse Kubernetes. NixOps 4, the experimental rewrite that is supposed to close the provisioning half of that gap, is still in development with no production release, so plan around Colmena-as-it-is, not NixOps-as-promised.

The rest of this post earns each row: a worked Colmena hive you can copy, what build-time reproducibility actually forecloses, an honest accounting of NixOps past and present, the three gaps that matter for a PaaS, and a decision checklist.

How Colmena Actually Works: A Hive, an SSH Connection, Nothing Else​

Colmena is a stateless NixOS deployment tool written in Rust, now maintained under the nix-community umbrella. The entire model fits in one sentence: you describe every machine on your workstation, Colmena builds each machine's system closure, copies it over SSH, and activates it. There is no control plane, no agent on the targets, no API server, and — the part that surprises Kubernetes operators most — no database recording what it did. Run the same command twice and the second run is a no-op because the target already matches, not because a state file says so.

A minimal hive managing two Hetzner boxes looks like this:

{
  meta = {
    nixpkgs = import <nixpkgs> {};
  };
 
  gateway = { name, nodes, ... }: {
    deployment.targetHost = "10.0.0.10";
    deployment.tags = [ "edge" ];
    networking.firewall.allowedTCPPorts = [ 22 80 443 ];
    services.tailscale.enable = true;
  };
 
  builder = { name, nodes, ... }: {
    deployment.targetHost = "10.0.0.11";
    deployment.tags = [ "ci" ];
    nix.settings.max-jobs = 8;
    services.prometheus.exporters.node.enable = true;
  };
}

With flakes, the same structure lives in your flake.nix as a colmenaHive output instead of a standalone hive.nix. The deploy loop is four steps: colmena build evaluates and builds every node's closure locally, colmena apply copies each closure to its targetHost and activates it, --on @edge limits a run to tagged hosts, and apply-local iterates on one machine without the SSH round trip. Deployments to all nodes run in parallel by default — fleet-wide rollout time scales with your slowest host, not your host count.

Statelessness is the feature, and it has a sharp edge worth naming now: Colmena deploys only to hosts already running NixOS. Its own migration guide states this outright — unlike classic NixOps, which could spin up EC2 instances, Colmena is kin to the SSH-only path and provisions nothing. Somebody, or something, has to install NixOS on the box first and keep it reachable. Hold that thought; it is the seam the whole comparison turns on.

What "No Drift" Really Means: Two Failure Scenarios, Two Models​

The claim that Nix eliminates drift sounds mystical until you walk a failure through both models. Take two mundane ones.

Scenario A: a well-meaning human edits a config file on the machine. On a NixOS box, /etc files managed by the system are symlinks into the Nix store — read-only, content-hashed paths. The edit either fails outright or lands somewhere the next colmena apply or reboot discards, because activation rebuilds system state from the derivation, not from what it finds. On a CAPI-managed Kubernetes node, the equivalent drift — a hand-edited kubelet flag, a manually added iptables rule — is invisible until something breaks; Kubernetes reconciles workloads, not node filesystems, and the fix is cordon, drain, and replace the machine from the template. Nix wins this one structurally: declared and installed are the same derivation, so there is no gap for a human to widen.

Scenario B: a daemon dies at 3 AM with no config change at all. Here the tables turn. Nothing in Colmena's model notices — convergence happens when you push, and between pushes the fleet is unobserved. There is no Colmena equivalent of a health check; a dead service stays dead until the next apply or until your separate monitoring pages someone. Cluster API plus Kubernetes is built for exactly this: kubelet probes restart the container, MachineHealthCheck remediates the node, controllers drive actual back to desired without a human pushing anything. Runtime reconciliation sees what build-time reproducibility cannot: the difference between "configured correctly" and "currently working."

NixOS does keep one runtime safety net worth knowing: every activation leaves previous system generations in the boot menu, so a bad push rolls back atomically with a reboot into the last known-good generation. That covers bad changes, not dead processes — the distinction that decides which model fits your fleet.

NixOps, Honestly: The sqlite Past and the Rewrite Present​

Classic NixOps was the ambitious one: a single deployment.nix describing both logical configuration and physical machines, with backends that provisioned EC2 instances, tracked every created resource ID in a local sqlite state file, and pushed closures the way Colmena still does. It worked, and its users paid two taxes for it. First, the state file: a local sqlite database with no locking, which production users learned to sync to S3 with wrapper scripts, is a single point of corruption for the whole fleet's memory. Second, the resource syntax: every cloud backend reimplemented its own resource model, so provider support rotted unevenly and maintenance never converged.

NixOps 4 — the experimental rewrite at nixops4/nixops4, backed in part by the Fediversity project — is a from-first-principles response to exactly those two taxes, and its stated goals read like a direct reply. Support both stateless and stateful deployments instead of mandating a state file. Fix the resource syntax and, more importantly, give resource developers a stable provider interface so backends stop rotting. Reuse the Nix module system and OpenTofu for provisioning instead of hand-rolling both. Rewrite the whole thing in Rust for a maintainable codebase.

That is a credible design — and it is explicitly marked in development, with no production release and no adoption numbers worth quoting. (If you see deployment-speed or drift-reduction percentages attached to NixOps 4 anywhere, ask for the methodology; there is no released artifact to measure yet.) The practical read for 2026: the rewrite validates Colmena's bet that state files were the problem, while aiming to reclaim the provisioning half Colmena deliberately surrendered. Until it ships, plan around Colmena-as-it-is: superb at converging existing NixOS hosts, silent about everything else.

Three Things Colmena Can't Do (That a CAPH Fleet Gets by Construction)​

1. Provisioning. Colmena needs NixOS already installed and reachable. A Cluster API fleet starts from provider credentials and a machine template: CAPH talks to the Hetzner API, creates the servers, bootstraps them into nodes, and replaces them on failure. With Colmena, that entire layer — ordering servers, imaging them, handling a dead box at the provider level — is your runbook, your Terraform, or your patience. For a handful of long-lived machines this is a one-time cost. For a platform that grows and shrinks node pools with demand, it is the whole game, and Colmena is not playing.

2. Scheduling and service discovery. A PaaS exists to run tenant containers that arrive unpredictably, need placing onto capacity, need finding each other, and need moving when nodes drain. Kubernetes gives you the scheduler, CoreDNS, and the Service abstraction for this; Cluster API gives you the fleet the scheduler runs on. Colmena gives you systemd units pinned to named hosts. You can absolutely run containers as NixOS-configured systemd services on fixed machines — many small platforms do — but placement, bin-packing, failover, and discovery become hand-written logic in your hive instead of platform primitives. The moment tenants outgrow named machines, you are rebuilding a worse scheduler.

3. Secrets with guardrails. Colmena ships a key-upload mechanism for getting secret files onto hosts, but no encryption story: combine it with agenix or sops-nix, which decrypt age- or GPG-encrypted secrets at activation time. That composes fine and the Nix community runs it widely — but notice what it is: two more tools, a key-distribution ceremony for every new host, and rotation by redeploy. Kubernetes' etcd-backed Secrets with RBAC, external-secrets operators, and short-lived projected tokens are not perfect, but they are one coherent system with an API your tenants' controllers can call. A Colmena fleet's secret story is files pushed from your laptop; there is no API for an agent or controller to request one.

Decision Guide: When the Nix Fleet Is Enough​

Choose the Colmena-managed Nix fleet when every statement below is true:

• Your machines are countable and mostly static — single digits to low tens, replaced rarely.
• Each machine's job is a fixed set of system services, not tenant code arriving over an API.
• You can provision and image hosts out-of-band (provider console, PXE, a golden NixOS image) without it dominating your week.
• Your team already thinks in Nix and will maintain the hive as carefully as application code.

Choose Cluster API and Kubernetes when any of these bite: tenants deploy containers, capacity must scale itself, a dead 3 AM process must heal without a human push, or an agent needs an API — not an SSH session from your workstation — to change fleet state.

The honest hybrid, and the one self-hosted PaaS builders actually land on: let each tool own its strength. NixOS machine images give Cluster API nodes the same bit-identical, drift-proof base that makes the hive trustworthy; Cluster API gives those nodes lifecycle, and Kubernetes gives tenant workloads scheduling. Immutable image baked by Nix, fleet reconciled by CAPI — build-time reproducibility below, runtime reconciliation above, each covering exactly the failure class the other cannot see.

The Push and the Loop Are Complements, Not Rivals​

Colmena's provocation survives contact with reality: an astonishing amount of fleet tooling exists to re-observe, re-compare, and re-converge state that a content-addressed build never let diverge in the first place. If your fleet's hard problem is configuration correctness across machines you already have, a control plane is overhead — a hive, an SSH key, and colmena apply genuinely suffice, and the absence of a state database is a reliability feature, not a missing one.

But configuration is only half of fleet operations. The other half is time: processes die, load shifts, tenants arrive, hardware fails, and none of those events wait for your next push. That half needs a loop, not a build — something watching, always. Know which half your fleet lives in, staff the other half deliberately, and be suspicious of any tool that claims one mechanism covers both.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Sources​

• Colmena — a simple, stateless NixOS deployment tool (nix-community)
• Migrating from NixOps/morph — Colmena manual (deploys to existing NixOS hosts only)
• NixOps 4 — experimental rewrite, status: in development (nixops4/nixops4)
• NixOps — declarative provisioning and deployment with a sqlite state file (NixOS/nixops)
• Cluster API — declarative Kubernetes cluster lifecycle management
• NixOS — generations and atomic rollback via nixos-rebuild