Kubernetes 1.36's PSI Metrics Go GA (Memory QoS Tiering Stays Alpha): What It Takes to Bin-Pack a Fixed Fleet

A node sitting at 50% requested capacity on paper can be stalling half its workloads in practice, and until Kubernetes 1.36 there was no kubelet-native metric that told you the difference. Requests and limits describe what you asked for. They say nothing about whether a task is actually waiting on a CPU core, a page of memory, or a disk queue right now. On an autoscaled hyperscaler cluster, that gap gets papered over — bin-packing runs a little loose, a new node spins up, nobody notices. On three or four owned Hetzner boxes with a dozen tenants split across them, there's no "spin up a new node." The gap is the whole game.

Kubernetes v1.36 "Haru" (April 22, 2026) ships two features aimed at exactly this gap, and they are not at the same stage of readiness — despite headlines that lump them together. Pressure Stall Information (PSI) metrics, KEP-4205, graduated to GA: the KubeletPSI feature gate is locked to true and can no longer be disabled. Memory QoS with tiered memory protection, KEP-2570, did not graduate to GA in 1.36 — it's still Alpha3, with Beta targeted for 1.37. One of these two features is safe to build production placement logic on today. The other is a canary-node experiment, not a fleet-wide bet, until at least the next release.

Here's the shape of what actually follows from that split. Wiring PSI into a scheduler's placement decisions — instead of just node CPU/memory requests — takes three concrete pieces: a per-node pressure score computed from PSI's avg60/avg300 windows, a soft scoring penalty that a scheduler extender applies on top of the built-in resource fit, and a hard admission gate that refuses new pods onto a node already in sustained full-stall. None of those three ship built into Kubernetes. All three are buildable today, on a feature that's GA. Memory QoS tiering, meanwhile, is worth turning on in Alpha form on a single canary node — but not as the thing your fleet-wide placement decisions depend on, because Alpha carries no upgrade-compatibility guarantee.

Why request-based bin-packing lies to you on a fixed fleet​

Say you're running four owned Hetzner AX102-class boxes (32 cores / 128 GiB each), packing a dozen tenant workloads across them by CPU and memory requests the way the default scheduler always has. Node 3 shows 16 of 32 cores requested — 50% allocated, plenty of declared headroom, the scheduler will happily land a thirteenth tenant there next.

What request-based allocation can't see: two of the six pods on Node 3 are bursty build jobs whose containers spend a lot of wall-clock time runnable-but-not-running, because they're contending for the same physical cores as everything else pinned there. cpu.some.avg60 on that node — the percentage of the last 60 seconds where at least one task was stalled waiting for CPU — is sitting at 22%. Node 1, by contrast, shows 70% of cores requested (less declared headroom) but cpu.some.avg60 at 3%, because its tenants' workloads happen to be spikier in memory than CPU. A scheduler that only reads requests puts the next tenant on Node 3, the node that's already stalling. A scheduler that reads PSI puts it on Node 1, the node with real slack.

That's not a hypothetical edge case — it's the ordinary state of a bin-packed fleet, because request/limit values are a ceiling a workload declares once, and PSI is a live read of what's actually happening to it. On a fleet that can add a fifth box whenever it wants, the mismatch is an efficiency question. On a fleet that has exactly four boxes and a waitlist of tenants, it's the difference between an oversubscribed-feeling platform and one that isn't.

What PSI actually gives you, concretely​

PSI is Linux kernel functionality (kernel 4.20+, CONFIG_PSI=y), not a Kubernetes invention — the kubelet in 1.36 just reads it natively instead of requiring a sidecar or manual /proc/pressure scrape. For each of CPU, memory, and I/O, the kernel tracks two states:

• some — at least one task is stalled waiting on that resource (early warning).
• full — every runnable task is stalled simultaneously (the node is functionally starved on that resource for everyone on it).

Each state carries four numbers: avg10, avg60, avg300 (percentage of the last 10/60/300 seconds spent stalled — moving averages, not point-in-time samples) and a cumulative total in microseconds. A real reading looks like this:

{
  "cpu": {
    "some": { "avg10": 0.74, "avg60": 0.52, "avg300": 0.21, "total": 35232438 },
    "full": { "avg10": 0,    "avg60": 0,    "avg300": 0,    "total": 0 }
  }
}

avg10 well above avg300 means a recent surge, not a sustained problem — worth a scheduling penalty, not an eviction. Any non-zero full value is the number to actually alert on: it means the node has a moment where nothing on it is making progress on that resource. In Kubernetes 1.36 these numbers are exposed at node, pod, and container granularity through the Summary API and, for scraping, through cAdvisor's Prometheus endpoint as container_pressure_cpu_waiting_seconds_total, container_pressure_memory_waiting_seconds_total, and container_pressure_io_waiting_seconds_total. Kubelet-side overhead for collecting all of this measured at roughly 0.1 cores (2.5% of a 4-core node) in the GA testing — cheap enough that there's no cost argument against turning it on, and no argument needed anyway, since the feature gate is locked on regardless.

Wiring it into placement: the three pieces Kubernetes doesn't ship​

This is the part the GA announcement is explicit about not including: no PSI-driven node taint, no PSI-based eviction, and no built-in scheduler plugin that reads pressure as a placement signal. An open enhancements discussion is still debating whether auto-tainting nodes on CPU PSI is even safe to standardize, given how easily a transient surge could get mistaken for sustained starvation. That debate is exactly why a platform operating its own fixed fleet shouldn't wait for upstream to ship this — the three pieces below are within reach as an out-of-tree scheduler extender, and the ambiguity upstream is worried about (surge vs. sustained) is solved by reading the right window, not by waiting for a taint controller.

1. A per-node pressure score. Poll each node's Summary API on the same cadence the scheduler already re-evaluates fit (every few seconds is enough — PSI's own averaging windows smooth out anything faster). Compute a score from cpu.some.avg60 and memory.some.avg60, weighted toward whichever resource dominates that node's tenant mix. This score sits alongside — not instead of — the existing requested-capacity calculation.

2. A soft scoring penalty in a scheduler extender. Kubernetes' scheduler framework supports exactly this: an extender or scheduling plugin that takes the default fit score and multiplies it down for nodes with elevated pressure, without hard-excluding them. A concrete rule that holds up across a realistic range rather than one convenient number:

3. A hard admission gate for sustained full pressure. The soft penalty handles "less good"; it doesn't handle "actively harming everyone already there." Any node reporting non-zero cpu.full.avg60 or memory.full.avg60 for more than a short grace window (long enough to rule out a transient blip, short enough to matter on a small fleet — a minute is a reasonable start) should be hard-excluded from new placements until it clears, independent of the scoring extender. This is the piece closest to what the upstream taint discussion is nervous about automating platform-wide — which is exactly why running it as your own admission check, scoped to your own fleet's tenant mix, sidesteps the generality problem that's stalled the upstream KEP.

None of this requires a Kubernetes patch. A scheduler extender reading the Summary API and returning adjusted scores is standard scheduling-framework surface, and the admission gate is a validating webhook reading the same data. What 1.36 changes is that the signal these three pieces need — real, per-node, per-resource stall data — finally comes from the kubelet itself, GA, on by default, instead of a bespoke /proc/pressure sidecar someone has to maintain.

Where Alpha Memory QoS fits anyway​

Memory QoS tiering is the enforcement half of this story, and it's worth understanding even at Alpha, because it changes how memory gets protected per QoS class rather than just measuring pressure. The kubelet gets a new memoryReservationPolicy field:

• None (default) — only memory.high throttling is active, using memoryThrottlingFactor (default 0.9). No hard reservation.
• TieredReservation — adds real cgroup v2 reservation, split by Pod QoS class:

A Guaranteed pod requesting 512 MiB gets memory.min set to exactly 536870912 bytes on that pod's cgroup — verifiable directly on the node (cat /sys/fs/cgroup/kubepods.slice/.../memory.min). This fixes a real problem from the original v1.27 version of Memory QoS: back then, every pod with a memory request got memory.min, so a Burstable pod requesting most of a node's memory could lock up nearly all of it as hard-reserved, starving the kernel and BestEffort workloads of headroom. Splitting Burstable into soft memory.low protection is what makes tiered reservation usable on a densely packed fleet instead of just Guaranteed-only clusters.

The catch is Alpha stability: no upgrade-compatibility guarantee, and it requires kernel 5.9+ (the kubelet logs a warning below that but won't block you — earlier kernels are exposed to a known memory.high livelock bug that 5.9 fixes). The two current metrics, kubelet_memory_qos_node_memory_min_bytes and kubelet_memory_qos_node_memory_low_bytes, are themselves Alpha-stability and exist for capacity planning, not alerting SLOs. The honest adoption path: enable it on one canary node in TieredReservation mode, watch how much headroom your Guaranteed tenants' hard reservations actually consume, and hold off on fleet-wide rollout until Beta lands the KEP is targeting for 1.37 — treating this the same way you'd treat any alpha feature gate on infrastructure you can't afford to get wrong.

A staged rollout for an owned fleet​

None of this needs to land at once, and most of it costs nothing to start:

1. PSI is already on. KubeletPSI is GA and locked-on in 1.36 — there's no flag to flip. Start piping container_pressure_*_waiting_seconds_total into whatever Prometheus stack already scrapes the fleet.
2. Build the scoring extender before the admission gate. The soft penalty is lower-risk and gives you signal on how often nodes actually cross the 15%+ threshold before you write a hard-exclusion rule that could refuse placements outright.
3. Canary Memory QoS tiering on exactly one node. Confirm kernel version, confirm memory.min/memory.low land where the docs say, and measure headroom impact before touching a second node.
4. Re-evaluate at 1.37, when Memory QoS tiering is expected to hit Beta and the upstream taint-controller debate in issue #5062 will likely have moved.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, orchestrated by Cluster API. Bin-packing a fixed fleet honestly, instead of pretending request-based headroom tells the whole story, is exactly the kind of operational detail an agent-operated platform has to get right before it's trusted to place a tenant's next deploy. Star the repo on GitHub or deploy your first app today.

Sources:

• Kubernetes v1.36: PSI Metrics for Kubernetes Graduates to GA
• Kubernetes v1.36: Tiered Memory Protection with Memory QoS
• Kubernetes v1.36: ハル (Haru) release announcement
• Understand Pressure Stall Information (PSI) Metrics
• Support memory qos with cgroups v2 (KEP-2570 tracking issue)
• Concerns on using CPU PSI pressure to taint nodes
• Kubernetes 1.36 Resource Management: What Actually Changed