8% CPU, 69% Overprovisioning: A 30-Minute Kubernetes Fleet Audit

Owning the servers does not make idle capacity free. It merely hides the waste from a cloud invoice and moves it into machines that draw power, occupy rack space, depreciate, and cannot host another tenant.

Cast AI's 2026 Kubernetes optimization report puts a provocative benchmark on that problem: average CPU utilization across its sample was 8%, while CPU overprovisioning reached 69%. Those percentages describe different layers of the stack, and neither diagnoses your cluster by itself. The useful response is to measure three quantities in your own fleet: what the nodes can provide, what Kubernetes has reserved through requests, and what workloads actually consume.

This article provides that audit. The analysis takes about 30 minutes if a Prometheus-compatible metrics system already retains at least 14 days of data. The worked example tests a four-node pool against three demand scenarios. Only the low-variability scenario clears the CPU test for shrinking from four nodes to three, and even that result is conditional on memory, pod capacity, placement, disruption, and a real drain rehearsal.

The 30-minute audit: capacity, requests, and actual use​

Start with a worker pool whose nodes have comparable capacity. Audit control-plane, GPU, storage, and general-purpose pools separately; combining them produces a ratio that no scheduler can act on. You need kubelet or cAdvisor usage metrics, node and pod state metrics, and 14 days that include a representative weekly peak. kubectl top is useful for a live spot check, but it cannot establish a historical percentile.

Collect these four CPU values first:

1. Allocatable: CPU the kubelet exposes to Pods after system reservations.
2. Requested: CPU that scheduled or pending Pods ask Kubernetes to reserve.
3. Fleet p95 usage: the 95th percentile of aggregate CPU consumed over the observation window.
4. Fleet peak usage: the maximum aggregate CPU consumed over the same window.

The following PromQL is a practical starting point. Metric labels vary between distributions, so verify each result against a known node and workload before trusting the total.

# Worker-pool allocatable CPU cores. Add your pool/role label selector.
sum(kube_node_status_allocatable{resource="cpu", unit="core"})
 
# CPU requested by Pending or Running Pods.
sum(
  kube_pod_container_resource_requests{resource="cpu", unit="core"}
  * on (namespace, pod) group_left()
  max by (namespace, pod) (
    kube_pod_status_phase{phase=~"Pending|Running"} == 1
  )
)
 
# Fleet-wide p95 CPU used over 14 days, sampled every five minutes.
quantile_over_time(
  0.95,
  (sum(rate(container_cpu_usage_seconds_total{
    container!="", image!=""
  }[5m])))[14d:5m]
)
 
# Fleet-wide peak CPU used over 14 days.
max_over_time(
  (sum(rate(container_cpu_usage_seconds_total{
    container!="", image!=""
  }[5m])))[14d:5m]
)

Recent kube-state-metrics documentation recommends the kube-scheduler's kube_pod_resource_request metric when it is available because it accounts more precisely for effective Pod requirements. The container-level query above remains common and portable, but check init containers, Pod-level resources, and any scheduler-specific overhead in your environment.

Turn the results into ratios with named denominators:

utilization             = p95 CPU used / allocatable CPU
request reservation     = requested CPU / allocatable CPU
unrequested capacity    = (allocatable CPU - requested CPU) / allocatable CPU
request-to-use slack    = (requested CPU - p95 CPU used) / requested CPU

Names matter. “We are 70% utilized” can mean 70% of node capacity is requested or 70% is actually executing. Those are different operational states.

Do the same for memory, using kube_node_status_allocatable, kube_pod_container_resource_requests, and container_memory_working_set_bytes. Prefer p99 and observed maximum working set for memory rather than copying the CPU p95 rule: CPU contention slows work, while exhausting memory can kill a process. The Kubernetes resource-metrics documentation notes that memory working set is a heuristic and may include cached file-backed pages, so combine it with OOM events and application behavior.

Finish with a decision worksheet. A CPU result is only the first gate.

Worked example: when can four nodes become three?​

Consider a self-hosted general-purpose pool with four 16-core workers. After operating-system and Kubernetes reservations, each exposes 15 allocatable cores, for 60 cores total. Running and pending Pods request 42 cores. Aggregate p95 usage is 13 cores and the observed peak is 18.

The first-pass ratios are:

• Request reservation: 42 / 60 = 70%.
• Unrequested capacity: (60 - 42) / 60 = 30%.
• p95 utilization: 13 / 60 = 21.7%.
• Request-to-use slack: (42 - 13) / 42 = 69%.

That last 69% is deliberately labeled. It happens to equal Cast AI's headline in this hypothetical example, but it has a different denominator from the report's capacity-to-request overprovisioning measure. A percentage without its formula is not an audit result.

Do not set new requests from the fleet-wide p95. Service peaks do not occur at the same time, and a small but latency-sensitive service still needs a minimum request. Calculate recommendations per Deployment, StatefulSet, Job class, and sidecar, then sum them with platform and DaemonSet overhead.

Suppose that workload-level exercise produces three plausible totals before a fleet change. The totals already include minimum floors and per-node system overhead. We then add 30% operating headroom:

Why is the failure capacity 30 cores rather than the new pool's normal 45? A three-node pool that must tolerate one unavailable worker has only two surviving nodes. N-1 turns a seemingly comfortable consolidation into a narrow gate. Under these assumptions, “CPU looks quiet” supports removing a node only for the low-variability model.

Now test the other dimensions. Assume each node has 58 GiB allocatable memory and 110 Pod slots. After rightsizing, memory p99 plus headroom is 92 GiB, and the pool will run 96 Pods including DaemonSets. Two surviving nodes provide 116 GiB and 220 Pod slots, so those gates pass.

The placement review finds one single-replica Deployment protected by a maxUnavailable: 0 PodDisruptionBudget. That is a drain blocker, not an invitation to use --force. The team raises the replica count to two, verifies anti-affinity permits the replicas on separate remaining nodes, and checks that no local persistent volume pins a Pod to the candidate node.

Finally, operators cordon the node and perform a normal drain with DaemonSets ignored. They predefine rollback thresholds: any sustained error-budget burn, p95 latency regression above 10%, OOM kill, CPU-throttling alert, or Pending Pod after the expected scheduling interval causes an uncordon and rollback. The drain completes in 12 minutes without crossing a threshold, and the pool remains stable through a representative traffic peak.

Only now is the low-variability case eligible for four-to-three consolidation. Eligibility is not a command to delete hardware. Keep the drained node available for rapid rollback before reducing a Cluster API MachineDeployment or repurposing the machine.

Why 8% utilization and 69% overprovisioning are not the same metric​

Cast AI's methodology page says its report analyzes tens of thousands of clusters across AWS, Google Cloud, and Azure, using data collected before those organizations enabled Cast AI automation. Supporting Cast AI material describes more than 23,000 production clusters. The report lists average CPU utilization at 8%, down from 10%, and memory utilization at 20%, down from 23%.

Its overprovisioning section describes another gap: provisioned capacity exceeds what workloads request. CPU overprovisioning rose from 40% to 69%, and memory overprovisioning reached 79%. In other words, the 8% figure concerns runtime consumption, while the 69% figure concerns the relationship between infrastructure capacity and requested resources. Other summaries sometimes compress these layers, which is exactly why your worksheet should retain all three raw totals.

Kubernetes creates this separation intentionally. The scheduler places a Pod according to requests, not current usage. The official resource-management documentation says it can reject a Pod when summed requests no longer fit even if actual CPU and memory use are low. That protects capacity for a later traffic peak, but an inflated request can make a node appear full and trigger node scale-out long before the silicon is busy.

The report is also a vendor-authored cloud baseline, not a randomized census of every Kubernetes cluster and not a measurement of your bare-metal fleet. Its sample is useful because the same scheduler mechanics apply on owned nodes. Its percentages are a warning to look, not evidence that your pool can lose 69% of its machines.

Right-size without buying an outage​

Treat rightsizing as a controlled reliability change, not a spreadsheet cleanup.

1. Observe a complete demand cycle. Fourteen days is a useful minimum for weekly workloads, but month-end processing, launches, or seasonal traffic may require longer. Record p95 and peak CPU, p99 and maximum memory, OOM kills, throttling, latency, and queue depth.
2. Generate recommendations without applying them. Run Vertical Pod Autoscaler in Off mode first. Kubernetes documents that this mode still writes target, lower-bound, and upper-bound recommendations to VPA status without changing Pods.
3. Separate CPU from memory policy. CPU is compressible and can often tolerate controlled overcommit. Memory is not; preserve more headroom, inspect working-set behavior, and never interpret “no OOM last week” as proof that a lower hard limit is safe.
4. Model HPA after changing requests. CPU-utilization HPA targets are calculated relative to requests. Lowering a request can raise the reported utilization percentage and add replicas, partially or entirely consuming the capacity you expected to reclaim.
5. Canary one workload class. Change recommendations for a low-risk Deployment, observe at least one peak, then expand by namespace or service tier. Keep explicit floors for startup bursts, JVM heaps, batch concurrency, and latency-sensitive services.
6. Rehearse the node loss. Cordon first, review a server-side drain dry run, then drain during a controlled window. Do not bypass a PDB or discard emptyDir data merely to make the test pass.
7. Change fleet capacity last. Observe the drained topology against predefined rollback thresholds. Only after it stays healthy should the infrastructure controller reduce desired machines or the operator reassign the server.

The Kubernetes VPA documentation also distinguishes RequestsOnly from RequestsAndLimits. That choice is consequential: changing a request affects scheduling and utilization-based autoscaling, while changing a CPU limit can introduce throttling and changing a memory limit can introduce OOM kills. Automation should not blur those controls.

Make efficiency a control loop, not a cleanup sprint​

A one-time audit decays as traffic, code, sidecars, and replica counts change. Put the same ratios on a weekly dashboard by pool, namespace, and workload. An example review policy might flag request-to-use slack above 50% for two full demand cycles, require an owner and expiry date for exceptions, and reopen a recommendation whenever p95 usage or replica count moves materially. Those are starting thresholds, not universal safe values.

Track the outcome in capacity units, not only percentages. “We lowered request slack” is abstract. “The pool now survives one-node loss on three machines instead of four, freeing one 16-core server for another tenant” connects the change to the economics of ownership. If a gate fails, the audit still creates value by naming the constraint: memory, disruption policy, topology, or genuine demand.

For a Cluster API-managed platform, machine lifecycle is declarative, but capacity judgment is not automatic. The controller can reliably make the fleet match a desired replica count; your measurements must establish that the desired count is safe. That feedback loop is the difference between owning efficient capacity and merely owning idle servers.

Bex.co is the open-source, AI-native Render alternative: push a Git repository and get a running HTTPS service on machines you own, with Cluster API managing the fleet lifecycle. Explore the project on GitHub.

Sources​

• Cast AI: 2026 State of Kubernetes Optimization report
• Cast AI: report methodology
• Cast AI: supporting 23,000-cluster description
• Kubernetes: resource management for Pods and containers
• Kubernetes: resource metrics pipeline
• Kubernetes: Vertical Pod Autoscaling
• Kubernetes: Horizontal Pod Autoscaling
• kube-state-metrics: Pod metrics reference