Your Cluster Runs at 8% CPU: Bin-Packing Defaults for a Self-Hosted Fleet That Can't Return Capacity

The average production Kubernetes cluster uses 8% of the CPU it has. Not 8% at night, not 8% in the dev environment — 8% average, across 23,000+ production clusters on AWS, GCP, and Azure, measured from direct telemetry between January and April 2026, before any optimization tooling was switched on. That's the headline of Cast AI's 2026 State of Kubernetes Optimization report, and the trend line is worse than the number: CPU utilization fell from 10% to 8% year over year, memory slid from 23% to 20%, and GPUs — the most expensive silicon in the building — sit at 5%. The share of clusters overprovisioning CPU jumped from 40% to 69% in a single year; 79% overprovision memory. Datadog's State of Cloud Costs frames the same reality from the billing side: roughly 83% of container spend pays for idle resources — 54 points of it idle cluster infrastructure, 29 points workloads requesting more than they use.

On elastic cloud, those numbers are a budget problem: embarrassing, recurring, fixable next month. On a self-hosted fleet — dedicated Hetzner boxes, colo racks, machines provisioned by Cluster API — they are something worse: stranded capacity you already paid for and cannot return. There is no instance to downsize, no bill that shrinks when you fix it. And the fix is not a FinOps platform. For a fleet you own, it's three defaults: what pods request, what they're limited to, and how the scheduler places them. Here they are, up front.

The Defaults Table: Requests, Limits, and Scheduler Settings for a Fixed Fleet​

This is the core of the post. Every row exists because one of the mechanisms in the next section pushes utilization toward 8% unless you set it deliberately.

The scheduler change is one small file. With kubeadm-style control planes (which is what a Cluster API KubeadmControlPlane gives you), mount a KubeSchedulerConfiguration:

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
profiles:
  - schedulerName: default-scheduler
    pluginConfig:
      - name: NodeResourcesFit
        args:
          scoringStrategy:
            type: MostAllocated
            resources:
              - name: cpu
                weight: 1
              - name: memory
                weight: 2

Memory gets the higher weight because it's the resource that actually runs out first on packed general-purpose nodes — and the one that fails hard when it does.

That's the deliverable. The rest of the post is why these defaults and not others, what the change is worth in money on owned hardware versus rented, and where packing has to stop.

How a Fleet Drifts to 8% Without Anyone Deciding To​

Nobody sets out to buy twelve cores and use one. The 8% cluster is the sum of four defaults nobody revisits:

• Copy-pasted requests. Helm charts ship cpu: 1000m because the author tested on a laptop and rounded up. The chart gets deployed 40 times. Cast AI's data shows the median workload requests far more than its p99 ever touches — and Datadog's 29-point "workload idle" share is exactly this line item.
• Fear stacked at every layer. The developer pads the request "to be safe." The chart maintainer pads the default. The platform team pads the LimitRange. Three reasonable 2× safety factors compound into 8×.
• The scheduler spreads by default. LeastAllocated scoring prefers the emptiest node for every placement. It's a defensible default for a cloud vendor — spreading reduces noisy-neighbor complaints — but on a fixed fleet it guarantees every node is a little bit busy and no node is ever full, which is the exact shape of the 8% histogram.
• "Headroom" reasoning never expires. Capacity added for a launch, a migration, a traffic spike stays in the fleet because removing it feels risky and nobody owns the decision. On cloud, unused nodes at least show up as a line item someone eventually questions. On owned machines, they're silent.

Each row in the defaults table counters one of these: measured requests kill the copy-paste, request-equals-limit memory kills the padding war, MostAllocated kills the spread, and the descheduler plus VPA recommendations make headroom something you see and re-decide, not something you inherit.

"Just Add Nodes": A Worked Comparison, Hetzner vs. Cloud​

Here's the asymmetry the title promises, in numbers. Take a deliberately typical small fleet: three dedicated Hetzner AX52-class machines (8 cores / 64 GB each) versus three comparable on-demand cloud instances (8 vCPU / 32–64 GB, m7i.2xlarge-class).

Two things fall out of this table, and they cut in opposite directions.

First, the direct dollar delta favors owned hardware overwhelmingly: the same wasteful 8% fleet costs about $1,000/month less on Hetzner — roughly $12,000/year for three nodes, before egress pricing makes the gap wider. Waste on cheap capacity is cheaper than waste on expensive capacity. This is why self-hosters can afford to be less aggressive than a FinOps team squeezing EKS.

Second — and this is the instinct the title calls worse — on cloud, "just add nodes" is a reversible operating expense; on owned machines it's a ratchet. When a cloud cluster feels tight, adding a node is a one-line autoscaler change you can undo when someone fixes the requests. When a Hetzner fleet feels tight at 8% actual utilization, the honest diagnosis is almost always that requests are lying and the scheduler is spreading — and adding a machine instead means: a purchase decision, provisioning lead time, another ~€65/month for years (dedicated boxes typically bill monthly but nobody decommissions them), and the original stranded capacity still stranded. You've converted a config review into capex. The cloud lets you paper over bad bin-packing with money every month; owned hardware makes you pay for the same mistake once, permanently, per mistake.

The sensitivity that keeps this honest: the delta scales with the driver, which is how wrong your requests are. If your fleet genuinely runs at 50–60% with correct requests, adding a node is simply buying capacity — do it. If it's memory-bound (the far more common real ceiling — remember, memory utilization is 20%, not 8%), fix memory requests first, because memory is what actually blocks scheduling. And there are three cases where adding a node is right even at low utilization: you need N+1 failure headroom you don't have, you need blast-radius isolation (a tenant or workload class that must not share nodes), or a single workload's peak genuinely exceeds what any current node can host.

Where Bin-Packing Must Stop: The Guardrails​

The strongest objection to everything above: "packed clusters fail harder." It's correct, which is why the target is a band — 50–70% of allocatable CPU requested per node — and not a score to maximize. Chasing 90% is the same mistake as tolerating 8%, in the other direction:

• N+1 is non-negotiable. On a three-node fleet, one node's death must be absorbable by the other two. That alone caps sustained packing near 66% — and it's the real reason the band tops out at 70%, not scheduler conservatism.
• Memory fails hard. A CPU-packed node degrades gracefully; a memory-packed node OOM-kills. Keep requested memory a step lower than requested CPU as a fraction of allocatable, and never let memory limits exceed requests on packed nodes.
• Descheduling must respect PodDisruptionBudgets. HighNodeUtilization evictions are voluntary disruptions; with PDBs in place, re-packing is a rolling, safe operation. Without them, it's an outage generator.
• Noisy neighbors return at high packing. No CPU limits means contention is arbitrated by request weight — which only works if requests are honest. That's the deal: you may drop CPU limits because you fixed CPU requests. One without the other reintroduces the problem.

The band's lower edge matters too: below ~50%, you're back to paying for idle steel; above ~70%, you're spending your failure headroom. The defaults table gets you into the band; the descheduler keeps you there as the fleet churns.

On a Cluster API Fleet, Capacity Is a Reviewed Change​

The quiet advantage of a declaratively provisioned fleet is that none of this has to be tribal knowledge. On a Cluster API fleet, node count is a MachineDeployment replica field in a git repo — which means "should we add a node?" becomes a pull request, and a pull request can demand evidence: VPA recommendations for the top ten workloads, per-node requested-vs-allocatable, actual p95 utilization. The 8% cluster survives in fleets where capacity decisions are verbal; it dies in fleets where a reviewer can ask "requests are at 8× actual — why are we buying steel instead of fixing them?"

That's also the operating model behind Bex.co, the open-source, AI-native Render alternative: apps deploy from a git push onto Cluster-API-provisioned machines you own, and the platform's state — including what every app requests and what its nodes actually run at — is machine-readable by design, for humans and for the AI agents operating the fleet. A bin-packing review is exactly the kind of chore an agent with access to utilization state should draft for you.

The Fleet You Already Own Is the Cheapest One to Reclaim​

The industry numbers are moving the wrong way — 10% to 8% CPU, 23% to 20% memory, overprovisioning nearly doubling in a year — because on elastic cloud, waste is survivable and optimization is someone's OKR for next quarter. A self-hosted fleet doesn't get that luxury, but it gets a better one: every point of utilization you reclaim is capacity you already paid for, returned to service by a config change. Measured requests, memory request-equals-limit, MostAllocated scoring, a descheduler, and a 50–70% target band — five decisions, one YAML file, and the difference between a fleet that's 92% stranded and one that earns its rack space.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.