Cilium Becomes the Default Kubernetes CNI Everywhere but AWS: What eBPF Networking Buys a Self-Hosted PaaS on Hetzner

Google made the choice for you: run GKE with Dataplane V2 — which every Autopilot cluster does — and you are already running Cilium. Microsoft built it in: "Azure CNI Powered by Cilium" is AKS's native eBPF data plane, kube-proxy replaced and all. Only AWS holds out, keeping its VPC CNI as the EKS default — and even there, EKS-Anywhere ships Cilium out of the box and plenty of production EKS teams run Cilium in chaining mode on top. Three clouds, one CNI, and the last holdout is only half holding out.

Here is the early verdict for the audience this blog cares about — operators running a self-hosted PaaS on their own Hetzner machines under Cluster API, who have always had a free choice of CNI: Cilium is worth adopting as your platform default, but for different reasons than the clouds picked it, and not for every fleet size. The clouds converged on Cilium for scale economics — eBPF service routing that stays O(1) while iptables chains grow linearly. A self-hosted PaaS gets a real share of that, but the bigger prizes are tenant isolation via identity-based network policy and Hubble's flow-level observability, which substitute for tooling you would otherwise build or buy. The costs are real too: more memory than Flannel, more moving parts than kube-router, and at 3-node scale the benchmarks say plain kube-proxy actually edges it out. This post walks the numbers on both sides and ends with a decision table.

"Everywhere but AWS": What Actually Converged​

The title claim deserves precision, because "default" means something slightly different on each cloud:

The project's trajectory matches the adoption curve. Cilium 1.19, released in February 2026, marked ten years since the first commit — roughly 2,900 commits from over 1,000 developers went into a release focused on hardened encryption, safer policy defaults, and large-cluster scalability. The v1.20 cycle followed immediately, with pre-releases through July 2026 adding Kubernetes Cluster Network Policy (KCNP) support, Gateway API BackendTLSPolicy, and a cilium-cni binary slimmed from roughly 77 MB to 16 MB.

For a self-hoster, the convergence itself is a signal worth reading: when GKE, AKS, and half of the EKS ecosystem standardize on one CNI, that CNI's talent pool, documentation, bug-discovery rate, and longevity all improve — and those are exactly the risks a small platform team weighs when picking infrastructure it will operate alone at 3 a.m.

What eBPF Buys, in Numbers​

The core technical difference is how Kubernetes Services get routed. kube-proxy in iptables mode appends rules per service; connection handling walks those chains, so cost grows with service count. Cilium's kube-proxy replacement implements the same Service abstraction with eBPF hash maps: O(1) lookup no matter how many services or endpoints exist.

Community and vendor benchmarks put consistent numbers on that difference:

Now the sensitivity analysis, because these numbers have a scale axis and it would be dishonest to hide it. In small-cluster tests — a 3-node cluster with a handful of services — the baseline kube-proxy setup slightly outperformed Cilium in both raw throughput and latency. With three nodes and twenty services, the iptables chain is so short that traversing it is effectively free, while Cilium's machinery carries fixed overhead. The eBPF advantage emerges as services and network policies accumulate: by the hundreds of services it is measurable, and by the thousands it is dramatic.

What does that mean for a PaaS? A platform hosting tenant apps accretes Services fast — every app deployment typically creates at least one, plus per-app policies if you isolate tenants properly. A 5-node fleet running 15 apps will not feel these benchmarks. A fleet hosting 50–100 apps with default-deny policies between them is exactly the regime where the iptables path starts costing real CPU and tail latency — and where the 8.9-vs-3.2 Gbps policy-throughput gap stops being a benchmark curiosity.

Hubble and the Policy Model: The Part a PaaS Actually Needs​

If raw routing performance were the whole story, a small self-hosted platform could shrug and keep Flannel. The stronger argument is that Cilium ships two things a multi-tenant PaaS otherwise has to assemble itself.

Identity-based network policy. A PaaS that runs each tenant app in its own namespace needs default-deny between namespaces — tenant A's web app must not be able to reach tenant B's Redis just because both live on the same flat pod network (Flannel enforces no policy at all; you would bolt on a second component just for this). CiliumNetworkPolicy enforces L3/L4 policy in-kernel via eBPF, and extends to L7 rules — allow only GET /api/* to a given service, restrict DNS lookups to expected names — without deploying a service mesh or sidecars.

Hubble, the observability layer. Every Cilium node runs a Hubble server exposing flow data over gRPC; a relay aggregates the streams cluster-wide. The flows are not raw packets — they are structured events with full Kubernetes context: pod, namespace, labels, service identity, and the policy verdict that allowed or dropped the traffic.

Concretely, when a tenant reports "my app can't reach its database," the diagnosis is one query:

hubble observe --namespace tenant-a --verdict DROPPED --last 100
# Aug  4 10:32:01.115 tenant-a/web-7f9c4 -> tenant-a/postgres-0
#   TCP Flags: SYN  DROPPED (Policy denied)

That output names the exact policy decision, the identities on both ends, and the timestamp — the difference between a five-minute fix and an afternoon of tcpdump on a node you had to SSH into. For a platform whose operators are increasingly AI agents rather than humans, the same property matters double: Hubble's gRPC API and structured flow events are a machine-readable answer to "what is the network actually doing," which is precisely what an agent needs before it can safely conclude anything about a networking incident.

What It Costs​

The honest ledger has three lines.

Memory. eBPF maps and Hubble state are not free. Comparative analyses put Cilium's extra memory versus Flannel at roughly 12–17 GB summed across a 100-node fleet — call it 120–170 MB per node. On a 5-node Hetzner fleet that is under 1 GB total, which on dedicated machines with 64 GB of RAM is noise. The often-cited "$300–400/month extra" figure comes from pricing that overhead at AWS instance rates — a cost that mostly evaporates on flat-price bare metal, which is a quietly funny footnote to the whole cloud-convergence story.

Kernel floor. Kube-proxy-free mode requires Linux v4.19.57 / v5.1.16 / v5.2 or newer. Current Hetzner Ubuntu and Debian images ship 5.15+ kernels, so this is a non-issue on fresh fleets — it only bites if you are dragging along pet servers with ancient kernels.

Operational complexity. This is the real cost. Cilium brings CRDs, an agent DaemonSet, an operator, optional Hubble relay and UI, and a large Helm values surface. Upgrades deserve reading the release notes, not blind helm upgrade. Flannel is a single binary doing VXLAN; kube-router is one process doing routing, policy, and service proxy. If your platform is a personal single-tenant fleet where nothing hostile shares the network, that simplicity is worth more than any benchmark — run Flannel and spend your complexity budget elsewhere. Calico remains a legitimate middle path if you want mature policy with an optional eBPF data plane and a stability-first reputation, though at that point you are operating a system of comparable weight without Hubble.

Wiring It into a Hetzner / Cluster API Fleet​

For a CAPI-managed platform (CAPH on Hetzner), the CNI is a day-0 addon: every workload cluster the control plane stamps out should come up with identical networking, not hand-installed afterwards.

The kube-proxy-free install reduces to a small set of Helm values:

helm install cilium cilium/cilium --version 1.19.x \
  --namespace kube-system \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=CONTROL_PLANE_ENDPOINT \
  --set k8sServicePort=6443 \
  --set routingMode=native \
  --set ipv4NativeRoutingCIDR=10.244.0.0/16 \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true

Two Hetzner-specific notes. First, skip kube-proxy at bootstrap (in CAPI, via kubeadmConfigSpec skip-phases) so Cilium owns service routing from the first boot. Second, choose the routing mode deliberately: on a Hetzner private network you can run native routing — pod CIDRs routed directly, no encapsulation overhead — while VXLAN tunneling is the safe default when nodes span networks you do not control. Delivery-wise, either a ClusterResourceSet or a Helm-based addon flow (Cluster API Addon Provider for Helm) binds the chart to a cluster label, so kubectl apply of a new Cluster manifest yields a machine set that boots straight into Cilium.

What does a git-push PaaS layer on top? Two things, both cheap once Cilium is underneath: a default-deny CiliumNetworkPolicy stamped into every app namespace at deploy time, with explicit allows generated from the app's declared service dependencies; and Hubble wired into the platform's ops surface, so "show me this app's denied flows" is an API call a dashboard — or an operating agent — can make.

The Verdict, and What's Coming​

The decision table, compressed:

Two forward-looking notes strengthen the Cilium call. KCNP support landing in the 1.20 cycle means the policy you write is tracking an upstream Kubernetes standard rather than a vendor dialect — portability insurance for policy, the way CAPI is portability insurance for cluster lifecycle. And the cloud convergence keeps compounding: every GKE and AKS cluster running Cilium in production is stress-testing the same code paths your Hetzner fleet runs, which is the kind of free QA a self-hoster should happily accept from hyperscalers.

The clouds picked Cilium because at their scale iptables was a tax on every packet. You get to pick it for better reasons: tenant isolation you can prove, flows you can query, and one less observability system to build — on machines where the memory overhead costs you nothing but the RAM you already own.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, on Cluster API-managed Kubernetes where networking choices like this one are yours to make. Star the repo on GitHub or deploy your first app today.

Sources​

• Cilium: Cloud Providers
• Cilium vs Calico on AKS: Which CNI Should You Actually Use?
• EKS vs GKE vs AKS: Best Managed Kubernetes Service in 2026
• InfoQ: Cilium at Ten Years — Cilium 1.19
• Cilium 1.20 Milestones (GitHub discussion)
• Cilium docs: CNI Performance Benchmark
• Cilium blog: CNI Benchmark — Understanding Cilium Network Performance
• Cilium vs Calico vs Flannel: Kubernetes CNI Comparison 2026
• Cilium docs: Kubernetes Without kube-proxy
• Cilium docs: Network Observability with Hubble
• Turing Pi: k3s Networking — Flannel vs Cilium vs Calico