Dora NodaThe most widely deployed per-pod power meter for Kubernetes recently threw away its measurement engine and built a different one. Kepler — the CNCF Sandbox project that attributes energy consumption to individual pods, namespaces, and nodes — replaced its eBPF-based architecture with a service-oriented design that reads hardware meters directly, starting in v0.10.0.
The project's own docs archive now labels everything written about the old pipeline as legacy. If you upgraded Kepler and kept the same Grafana dashboards, the numbers on them are coming from a different instrument than the one you validated.
That is the entire argument of this post: re-validate before you showback. The rewrite is a reasonable engineering trade — lower privileges, less overhead, easier to deploy — but it changes the accuracy contract in ways that matter the moment energy numbers leave the observability dashboard and enter tenant-facing territory like showback reports or sustainability pages. Here is the verdict up front:
| Question | Answer |
|---|---|
| Did v0.10 get cheaper and easier to run? | Yes — no eBPF, reduced privileges, reads /proc, /sys, and hardware meters |
| Did per-pod attribution get more accurate? | No — that was never the goal; temporal resolution and kernel-level attribution went down, not up |
| Can it measure on bare metal with RAPL/hwmon/Redfish/NVML? | Yes — that path is the whole point of the rewrite |
| Can it measure inside a typical cloud VM? | Usually no — no RAPL in the guest means it falls back to its estimator |
| Is the estimator good enough for showback? | Red Hat's own docs say the estimator is not supported — treat that as the industry answer |
| Do the Red Hat and Kubeflow endorsements certify the new numbers? | No — they certify packaging and dashboards; accuracy re-validation is still your job |
The rest of this post fills in each row: what the rewrite changed, what losing eBPF costs in accuracy terms, why virtualized nodes are the trap, how to read the enterprise signals correctly, and a six-step runbook a platform team can execute in a week.
What v0.10 actually changed: a new measurement chain
Kepler began as a joint Red Hat and IBM Research effort to answer a simple question: how much energy did that pod consume? The original pipeline had three stages. First, eBPF programs attached to kernel tracepoints collected per-process CPU performance counters, IRQ behavior, and cgroup identity at high frequency. Second, hardware readings — Intel RAPL energy counters via powercap, and later BMC and GPU telemetry — established the node's total power draw.
Third, a power model (the Ratio Power Model, backed by a trained model server) split the node's dynamic energy across processes in proportion to their measured usage. It separated always-on idle power from load-dependent dynamic power, then rolled processes up into containers, pods, and namespaces exported as Prometheus metrics.
The v0.10 rewrite keeps the second stage and rebuilds the other two. eBPF is gone from the main branch entirely — a February 2026 cleanup commit removed even the stale license references to the eBPF code, and the documentation project has an open tracker item to purge eBPF references from the power-attribution docs.
In its place: a service-oriented design that reads process accounting from /proc and /sys, keeps the direct hardware-meter readers (RAPL, hwmon sensors, Redfish BMC, NVIDIA NVML — all visible as fresh 2026 feature work in the release history, including graceful degradation when a BMC is unreachable), and narrows attribution to a CPU-focused model instead of the old multi-component breakdown.
One more documented gap: the trained ML model server does not yet plug into the rewritten pipeline, per the project's own docs tracker. The estimator that fills gaps when hardware meters are absent is still there — and still an estimator.
None of this is a criticism of the rewrite. Shipping eBPF programs into other people's kernels is a support burden: privileged DaemonSets, kernel-version compatibility matrices, security review friction on every cluster. Reading /proc runs anywhere. The tradeoff was deliberate — deployability and overhead in exchange for measurement fidelity — and for cluster-level trend dashboards it is arguably the right call. The problem is only that the dashboards look identical before and after, so teams inherit the old confidence with the new numbers.
What losing eBPF costs: resolution, attribution, and the fork that proves it
Three accuracy-relevant things change when sampling moves out of the kernel into userspace polling of /proc and /sys.
First, temporal resolution drops. eBPF tracepoints observe scheduling and counter events as they happen; /proc polling sees whatever accumulated between scrapes. Short-lived processes — the bursty sidecars, init bursts, and sub-second agent sandbox runs that increasingly populate shared fleets — smear across sampling windows or fall between them.
A 2026 empirical study of RAPL-based tooling overhead (Diamond and Stoico) makes the other side of this bargain explicit: high-frequency energy measurement itself costs measurable overhead, which is exactly why upstream accepted coarser sampling. You cannot have both free and fine-grained; v0.10 chose cheap.
Second, attribution granularity coarsens. The old pipeline attributed energy with kernel-observed per-process performance counters feeding a trained model; the new pipeline does active-CPU-proportional attribution from userspace accounting. When every tenant's workload is CPU-bound in the same way, proportional splitting is fine. When the fleet mixes AVX-heavy inference, memory-stall-bound caches, and idle-but-resident agents, CPU-time proportionality assigns energy by a proxy that correlates unevenly with actual joules. The Ratio Power Model's idle/dynamic split still exists, but the dynamic portion is now divided by a blunter instrument.
Third, and most telling: someone already forked over this exact disagreement. Tycho, an accuracy-first container-level energy exporter, is explicitly based on Kepler v0.9 — the last of the old architecture. Its stated motivation is that upstream is "evolving toward reduced privileges and lower overhead, de-emphasizing high-frequency eBPF-based collection," a shift that "limits temporal resolution and attribution."
When practitioners who need defensible numbers pin themselves to your pre-rewrite release and maintain a fork, that is independent confirmation of what the tradeoff costs. It does not mean v0.10 is wrong; it means v0.10 optimized for a different consumer than the team doing per-tenant showback.
The VM trap: where the estimator quietly takes over
Here is the failure mode that actually bites fleets. Kepler's measurement path needs hardware counters: RAPL zones under /sys/class/powercap, hwmon sensors, a reachable Redfish BMC, or NVML on GPU nodes. Inside a typical virtualized guest, none of the CPU-level sources exist.
Cloud hypervisors do not pass RAPL through — Amazon has stated this plainly for EC2, Lambda, and Fargate in a security bulletin, and researchers have confirmed that newer KVM-based EC2 instance types no longer expose RAPL metrics at all, leaving bare metal as the only reliable way to read them. This is not an AWS quirk; it is the standard posture of multi-tenant virtualization, where exposing host power telemetry to guests would leak host-level information.
When the hardware sources are absent, Kepler does not fail loudly. It falls back to its estimator — and this is where Red Hat's documentation becomes the most useful sentence in this whole debate. The OpenShift power monitoring docs state that when Kepler cannot obtain hardware power consumption metrics, the component power-source column displays "estimator," which is not supported.
Red Hat ships the dashboards, supports the operator, and explicitly declines to stand behind estimated numbers. A platform team reaching for stricter guarantees than Red Hat offers its own enterprise customers should pause.
For a fleet on Hetzner hardware, this draws a bright line worth checking rather than assuming. Bare-metal dedicated nodes expose RAPL through the standard powercap interface, and enterprise chassis may offer BMC telemetry where the provider grants access — that is Kepler's home turf. Cloud VMs, behind a KVM hypervisor like every other cloud guest, should be expected to hide RAPL until proven otherwise. The check takes ten seconds on any node:
ls /sys/class/powercap/intel-rapl* >/dev/null 2>&1 \
&& echo "RAPL visible: Kepler can measure" \
|| echo "no RAPL: Kepler will estimate"Run it across your node pools before you believe a single per-pod joule. A mixed fleet — dedicated machines for GPU and dense tenants plus cloud VMs for burst — will produce a dashboard that silently mixes measured and estimated numbers under identical metric names.
That mixture is the thing to fear: not that estimates exist, but that nothing in the default dashboard tells you which rows are which. Assert the power source per node, or your showback report is blending two instruments and labeling them one.
Reading the enterprise signals correctly: preview means packaging, not proof
Two institutional endorsements arrived around the same time as the rewrite, and both are easy to misread as accuracy certifications. They are not.
Red Hat's power monitoring for OpenShift — Kepler packaged as an operator with two console dashboards (Overview and Namespace under Observe) — entered Developer Preview and has since moved through Technology Preview releases. The 0.5 release notes are instructive about what "productization" actually means here: the Kepler custom resource was deprecated in favor of a PowerMonitor CR, dynamic RAPL-zone detection and improved VM/container/pod detection landed, and Redfish configuration support was removed.
That is an operator maturing its API surface and support matrix, which is valuable work — and orthogonal to whether the rewritten attribution pipeline reproduces the old pipeline's numbers. Red Hat validating the deployment story while marking the estimator unsupported is the vendor telling you exactly where its confidence ends.
The Kubeflow signal is similar in shape. The community-distribution project carries an open proposal for an opt-in observability stack that would bundle Kepler's manifests as an opt-in sub-component, scrape per-pod CPU-plus-GPU energy through a ServiceMonitor, and ship provisioned Grafana dashboards — with the proposal noting the stack already runs in production on a KubeFlow cluster at CERN.
That is meaningful adoption evidence for Kepler-as-observability-plumbing on training infrastructure. It is not a controlled comparison of v0.10 attribution accuracy against wall power, and an opt-in kustomize overlay does not perform one. Borrow the packaging; do your own validation.
The general rule: enterprise preview status certifies that smart people made the software installable, dashboarded, and supportable. It never certifies that a rewritten measurement pipeline agrees with the instrument it replaced. Calibration is always the operator's job, because only the operator knows their nodes, their kernels, and their workload mix.
The re-validation runbook: six checks before showback becomes the default
This is the core deliverable — the week-long procedure that turns "we upgraded Kepler" into "we trust these numbers per tenant." Do it once per hardware profile in your fleet, and repeat the cheap parts on every Kepler minor-version bump, because the rewrite is still settling (Redfish handling, GPU attribution, and the model-server integration all moved within the last year).
1. Cross-check node totals against an independent meter. Pick one dedicated node per hardware profile, put a realistic mixed workload on it, and compare Kepler's node-level total against the BMC power reading or a wall meter over 24 hours. If the totals disagree by more than your showback tolerance — pick one in advance, say 10% — stop here. Per-pod splits of a wrong total are decoration.
2. Assert the hardware power source on every node. Query which power source each Kepler instance is actually using and alert on anything reporting the estimator. The rule is absolute: measured nodes feed showback, estimating nodes feed only capacity trendlines. Automate the /sys/class/powercap check from the previous section as a node-labeling job so the scheduler of truth is infrastructure, not memory.
3. Pin the Kepler version and record the pipeline. Note the exact release, because "Kepler says" is meaningless across the v0.10 boundary and across minors that touch attribution. Keep one node on the previous version during upgrades and diff the per-pod series for your five highest-energy tenants before rolling forward.
4. Test your actual workload mix, not a benchmark. CPU-proportional attribution is most wrong exactly where fleets are most heterogeneous: inference with AVX/GPU bursts, memory-bound caches, and idle-but-resident agent sandboxes. Include one of each in the validation workload, and check the splits against per-cgroup CPU accounting you already trust. If bursty tenants systematically drift, widen the error bars for that class instead of pretending the point estimates hold.
5. Separate showback from billing in writing. Publish energy numbers to tenants as informational with stated error bounds — "measured on bare metal, ±10% against BMC" reads very differently from a bare joule count — and keep estimated-node numbers out of anything a customer could mistake for a meter. The moment an estimate appears on an invoice, you have adopted a liability no vendor in this story will share.
6. Re-run checks 1–4 on every minor bump. The pipeline is under active reconstruction, and each release can move the numbers without moving the dashboards. A quarterly half-day of re-validation is the entire cost of never having to explain to a tenant why their energy bill changed on a release day.
Measure like a lab, bill like a utility
Kepler's rewrite is good news for almost everyone: more clusters can now run power observability, with fewer privileges and less overhead, feeding the same Prometheus-and-Grafana plumbing that OpenShift productizes and Kubeflow proposes to bundle. Standardize the collection now — per-pod energy visibility is worth having even at coarser resolution.
But treat the numbers as what the new instrument actually is: cheaper, broader, and unproven at the per-tenant granularity where money and reputation attach. Run the six checks, publish the error bars, and let the showback default wait until your own meters agree with Kepler's. The dashboard will look the same either way. Your confidence should not.
Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.
