Skip to main content

Show HN Promised Instant Kubernetes With KVM and gRPC — the Hard Part Was Never Provisioning

13 min readDora NodaDora Noda
Share
On this page

A recent Show HN pitches exactly what every tired platform team wants to hear: production-ready Kubernetes in 5–10 minutes, spun up by agents talking gRPC, running on KVM virtual machines, with Postgres, Kafka, and monitoring already installed. The verdict first, because the rest of this post is just the receipts: the 5-minute number is real and also beside the point. Day-0 provisioning speed is a solved problem with multiple answers. The thing that eats platform teams alive is everything after minute ten — and a faster provisioning protocol does nothing about that.

Pitch claimWho already answers itVerdict
5–10 minute cluster provisioningkubeadm + any automator; Cluster API; Talos; managed cloudsSolved many times over — speed was never the moat
Declarative machine lifecycle (add/remove/replace nodes)Cluster API MachineDeployments + provider (e.g. CAPH on Hetzner)Standardized; reconciliation loop included
Upgrades without rebuild surgeryCAPI rolling + v1.12 in-place updates and chained upgradesStandardized, still operator-attended
Drift repair and self-healingCAPI controllers + MachineHealthCheck + GitOpsStandardized; the controllers never sleep
One-click Postgres/Kafka/monitoring catalogHelm charts, operators, Carvel/Crossplane bundlesAvailable but unopinionated — nobody ships your matrix
Cross-service version-compatibility matrixNobody's shared artifact; every vendor hand-rolls itGenuinely unsolved — this is the real work the pitch names
Certs, CNI, storage, WireGuard defaultscert-manager, Cilium, Longhorn/OpenEBS — assembled by youSolved as parts, unsold as a tested whole
Firewall-free agent connectivityAgent-initiated gRPC streams; WireGuard/Tailscale overlaysNice engineering, not a differentiator

The honest summary: the left column splits cleanly. Machine lifecycle is standardized territory. The service bundle is genuinely hard, unsolved-in-common territory. And gRPC over KVM is an implementation detail that determines neither.

What the box actually contains

The Show HN post (RunOS, by dib85 on Hacker News) describes a system built over a year with two agent types. Server agents run on VM hosts and hold agent-initiated gRPC bidirectional streams back to the backend; when a user clicks "Create Cluster," the backend picks agents and sends provisioning commands. Node agents run on each Kubernetes node and handle cluster operations, monitoring, and service installs. The key networking insight is genuinely neat: because the agents dial out, there is no firewall configuration and no public-IP requirement on the hosts.

Underneath, the stack is deliberately boring, and the author says so outright. KVM virtual machines via libvirt's Go bindings, chosen for Ubuntu maturity, isolation, and GPU passthrough for AI workloads like Ollama. Ubuntu Cloud 24.04 images boot in 30–60 seconds.

Kubernetes itself is bootstrapped with kubeadm plus Cilium — the same two primitives half the industry uses. WireGuard is managed at the OS level rather than inside Kubernetes, so the same mesh secures SSH access, survives a Kubernetes outage, and keeps troubleshooting layers separated. Storage is OpenEBS plus Longhorn. Total time from click to a cluster running Postgres, Kafka, and Prometheus: 5–10 minutes.

Then comes the most honest paragraph in the whole post, and the one this article keeps coming back to: "The hardest problem? Keeping 20+ services compatible across updates." The catalog — PostgreSQL, MySQL, ClickHouse, Kafka, RabbitMQ, MinIO, Harbor, Traefik, Grafana, Prometheus, Ollama, LiteLLM, Open WebUI, and more — is installed via a mix of Helm charts, operators, and custom YAML. The real work, by the author's own account, is maintaining compatibility matrices and testing every combination.

Deployment comes in two flavors: a managed cloud with fixed 8 CPU/16 GB instances, or bring-your-own-node on any hardware. Agent code is promised open source; one early company reportedly runs three production clusters.

Nothing in that inventory is suspicious. All of it is legible engineering. The question is only which parts of it constitute news — and which parts the ecosystem standardized years ago.

Claim by claim: the lifecycle half is settled science

Take the machine-lifecycle rows first, because this is where "we built provisioning" most directly collides with "provisioning already has a declarative API."

Cluster API exists precisely so that creating, scaling, upgrading, and deleting Kubernetes-conformant clusters is a manifest change, not a provisioning ticket. A MachineDeployment expresses "three workers of this shape"; the controllers converge reality toward it, including replacing failed machines without a human composing gRPC calls at 3am. KubeadmControlPlane does the same for control-plane members.

Infrastructure providers like CAPH extend the identical contract to Hetzner machines — including bare metal — so "declare desired state, let controllers reconcile drift away" holds on owned hardware, not just on hyperscaler APIs. Pair it with a GitOps tool and the fleet's entire intended state lives in versioned YAML with review, rollback, and audit attached.

Upgrades, the traditional graveyard of hand-built provisioners, keep getting less disruptive on the standardized path. Cluster API v1.12, released in January 2026, introduced in-place updates through update extensions — changes applied to existing machines without deleting and recreating them — alongside chained upgrades that sequence dependent updates safely. That lands on top of the long-standing rolling-update machinery both KubeadmControlPlane and MachineDeployments already had. Nobody sane calls CAPI upgrades effortless; pre-flight audits against breaking changes and provider compatibility still consume real operator hours. But the mechanism is a shared, reviewed, multi-vendor artifact with a changelog, not a bespoke agent command set whose upgrade path is "trust our backend."

Drift repair is the quietest row and the most lopsided. A controller loop that continuously reconciles observed state toward declared state is simply a different species from an agent fleet awaiting commands. MachineHealthCheck notices the dead node and remediates it whether or not anyone is watching a dashboard. An imperative provisioner can bolt on equivalent watchers — node agents already report monitoring data, so the raw signal exists — but every watcher is new code the team writes, tests, and maintains forever, reproducing behavior the CAPI controllers ship by default.

None of this says building a provisioner is wrong. It says the lifecycle half of the pitch is competing against a CNCF-governed standard with a provider ecosystem, and "instant" is not a dimension where the standard is weak: CAPI-backed fleets routinely report cluster provisioning in the tens of minutes including control-plane bootstrap, and the VM-boot seconds the pitch advertises are a property of KVM and cloud images, not of the orchestration protocol above them.

The bundle half is where the pitch actually lands

Now flip to the rows where the conceded ground reverses — because the Show HN's real complaint was never "VMs boot too slowly." It was: "every team rebuilds the same networking, certificates, monitoring, databases, storage."

That complaint is accurate, and Cluster API does not answer it. CAPI provisions machines and joins them to clusters; it has no opinion about which Postgres operator version is compatible with which Kubernetes minor, which Longhorn release survives the CSI changes in the next upgrade, or whether your Traefik, cert-manager, and Cilium trio interoperates on the kernel you just rolled out. The ecosystem gives you excellent parts — Helm charts, operators, cert-manager, a choice of CNIs and storage systems — and then hands you the assembly, testing, and compatibility matrix as an exercise. Surveys keep confirming the pain: D2iQ found 96% of organizations hitting challenges getting containerized apps to day-2 operations, with Kubernetes add-ons singled out as a major source of pain, and only about four in ten organizations report all their Kubernetes apps ever reaching day-2 environments at all.

This is the context in which the author's "version management hell" paragraph deserves respect rather than a rebuttal. Maintaining a tested matrix across 20+ services and every supported Kubernetes version is genuinely the work. It is also work with brutal economics: it is invisible when done right, it never finishes because Kubernetes ships three minor releases a year and every dependency moves independently, and its output — a private compatibility spreadsheet plus a pile of pinned charts — is nearly impossible to share across vendors. Nobody's shared artifact covers "ClickHouse operator X with Longhorn Y on Kubernetes 1.36 on kernel Z," and that specificity is exactly what production needs.

So credit where due: a batteries-included catalog with a vendor willing to test the combinations is a real offer, and it is the offer CAPI-plus-GitOps structurally cannot make. The standard gives you the machine lifecycle; the catalog gives you the Tuesday afternoon back. The catch is only that the catalog is also where lock-in quietly re-enters through the side door the pitch claims to close. The further your Postgres, Kafka, networking, and storage drift toward one vendor's tested matrix, pinned charts, and agent-managed WireGuard configs, the more "bring your own node" means "bring your own node to our control plane, our agent protocol, and our backend's orchestration state." The announced open-sourcing of the agent code would mitigate this materially — self-hostable agents plus documented state would make the exit door real — but until it ships, the honest framing is managed-platform convenience with self-hosted hardware, not the absence of a platform dependency.

Why the protocol isn't the point: the 3am test

Strip away the bundle, and the remaining technical claim is architectural: agent-initiated gRPC streams driving KVM provisioning. Evaluate it the way operators evaluate everything — not by how the demo goes, but by what happens when a node dies at 3am with nobody awake.

In the imperative model, recovery is a sequence of commands that must be composed, ordered, and retried by something: detect the dead node, select a healthy host, provision a replacement VM, bootstrap it, join it, re-establish WireGuard, reschedule the workloads, verify health. Every step is code the platform team owns. The failure modes are partial execution (VM exists but never joined), ordering bugs (workloads scheduled before storage attached), and state skew between the backend's database and reality — the classic distributed-systems failure set that command-driven orchestration must solve from scratch, per platform, forever.

In the declarative model, recovery is a property of the loop: desired state still says three healthy workers, observed state says two, and the controllers act on the difference — MachineHealthCheck flags the failure, the MachineDeployment converges a replacement through the provider, kubelet and CNI rejoin it, the scheduler rebalances. The failure modes are different and, crucially, shared: reconciliation-edge bugs get found by every CAPI user at once and fixed upstream once. Your 3am incident becomes an upstream issue with five reproducers instead of a private postmortem about your agent queue.

This is also why "gRPC over KVM" misidentifies the layer where leverage lives. KVM is a fine hypervisor choice — boring, libvirt-scriptable, GPU-passthrough-capable — but the VM-boot seconds are available to any orchestrator sitting above libvirt, CAPI's KVM providers included. Agent-initiated streams are a fine connectivity choice — outbound-only, firewall-friendly.

But WireGuard or Tailscale overlays plus standard tooling deliver the same topology without a custom protocol. Neither choice touches the actual cost center, which the author's own post names: the compatibility matrix. You cannot gRPC your way out of testing ClickHouse against Longhorn.

There is one more cost the protocol framing hides: every bespoke agent is a second codebase with its own release cadence, its own upgrade story, and its own security surface, running privileged on every host and every node. CAPI providers concentrate that privileged code into versioned, RBAC-scoped controllers with a shared security-review audience. A from-scratch agent fleet starts that review count at one team. For early-access software running on customer hardware, that is the correct thing to price into the "5 minutes to a cluster" headline: the minutes are cheap; the trusted-computing-base expansion is the actual invoice.

A distribution-and-defaults problem, not a protocol problem

Reframe the original complaint once more: "we wanted the control of self-hosting without weeks of setup." Weeks of setup come from assembling unopinionated parts — picking a CNI, a storage system, a cert story, a Postgres operator, a monitoring stack — and then discovering their incompatibilities one production incident at a time. That is a packaging and defaults problem. It is solved by curated, tested, versioned distributions with someone's name on the matrix, the way Linux distributions solved "every team rebuilds the same userspace" two decades ago. It is not solved by a new provisioning transport, because the transport was never the bottleneck: kubeadm has bootstrapped clusters in minutes for years, and CAPI turns those minutes into a reconciled fleet.

For a team choosing today, the decision checklist follows directly:

  • If your pain is machine lifecycle on owned hardware — provisioning, scaling, upgrades, drift, node replacement — adopt the standardized loop (Cluster API plus a provider like CAPH, driven through GitOps) before writing or adopting any bespoke provisioner. That territory is settled.
  • If your pain is the service bundle — tested Postgres/Kafka/monitoring/storage combinations with a compatibility matrix — evaluate batteries-included platforms on the strength of their matrix and their exit story (open agents, exportable state, standard charts underneath), not on their provisioning minutes.
  • If a pitch leads with its transport protocol, ask what it does at 3am without a human, who maintains the version matrix, and what the migration off the platform looks like. The answers to those three questions are the product; the protocol is the brochure.

The Show HN deserves its hearing for naming the real wound — version management hell is the most honest phrase in the post — and any team that tests the matrix so you don't have to is selling something of genuine value. Just don't mistake the delivery van for the cargo.

Instant clusters were never the hard part. The hard part is the second year: the third Kubernetes upgrade, the Longhorn CVE, the Postgres major version, the drift nobody noticed. Whoever owns that calendar owns your platform, whatever protocol the agents speak.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Sources

  • Show HN: "We built instant Kubernetes provisioning with KVM and gRPC" (RunOS) — Hacker News discussion, news.ycombinator.com/item?id=45927988
  • "Cluster API v1.12: Introducing In-place Updates and Chained Upgrades" — kubernetes.io/blog/2026/01/27/cluster-api-v1-12-release/
  • "94% of Organizations Run Into Kubernetes Challenges" / D2iQ day-2 survey coverage — DEVOPSdigest; only 42% of organizations report all Kubernetes apps reaching day-2 — ZDNet/CNCF survey coverage
  • "Day 2 Operations with Kubernetes" whitepaper — Nine.ch; "Kubernetes Isn't Enough for a Production-Ready Platform" — The New Stack
  • "Cluster API and GitOps: the key to Kubernetes lifecycle management" — CNCF on-demand webinar

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex