Kubernetes 1.36 Resizes Pods In Place — Until It Doesn't: The Two Boundaries Your Autoscaler Still Has to Respect

Kubernetes 1.36 turned on pod-level in-place resizing by default, so a running Pod's whole CPU and memory budget can now grow or shrink without a restart. That sounds like the end of evict-and-recreate vertical scaling. It is not. Two hard boundaries survive every version of this feature: a resize that would change the Pod's QoS class is rejected by admission, and a memory shrink that dives below what the container is actually using is blocked by the kubelet rather than allowed to OOM-kill. Everything else in this post is the consequence of those two sentences: what each boundary rejects, why the kernel and the API server insist on it, and the exact fallback — recreate, not resize — your autoscaler still needs for each case.

What 1.36 actually shipped: one budget for the whole Pod​

Container-level in-place resize (KEP-1287) is already GA: you can PATCH a running Pod's resize subresource and the kubelet adjusts cgroup limits without restarting anything. What 1.36 adds, behind the InPlacePodLevelResourcesVerticalScaling gate now in beta and enabled by default, is the same trick one level up — the aggregate Pod-level resource budget in .spec.resources.

Pod-level resources exist for Pods whose containers share a collective pool instead of each carrying its own fixed slice — the canonical case is a Pod with a sidecar, where the app and its proxy draw from one shared budget. Before 1.36, resizing that aggregate boundary meant recreating the Pod. Now it goes through the same resize subresource:

kubectl patch pod my-app --subresource resize --patch \
  '{"spec":{"resources":{"requests":{"cpu":"400m","memory":"512Mi"}}}}'

Note the kubectl version floor: the --subresource=resize flag needs kubectl 1.32 or newer. And the scope is deliberately narrow — only CPU and memory, only within the rules below. The feature changes how a resize is applied, not which resizes are legal. Legality is where the two boundaries bite.

Boundary 1: the QoS class is set at creation and resize cannot move it​

Every Pod is born into one of three Quality of Service classes — Guaranteed, Burstable, or BestEffort — derived from its requests and limits. That class is computed at creation and frozen. Any resize that would flip it is rejected at admission with an Invalid error, per Pod, while in-place stays enabled for everything else.

The rules that matter in practice:

• Guaranteed means every container has requests == limits for both CPU and memory. Shrink requests without shrinking limits by the same amount and the Pod would become Burstable — so the PATCH is rejected.
• Burstable means requests and limits are not equal for both resources at once. Grow requests until they equal limits on both CPU and memory and the Pod would become Guaranteed — also rejected.
• BestEffort (no requests or limits at all) cannot gain resources through resize into another class either.

Here is the most common way teams hit this wall. A tenant's web app runs Guaranteed at 500m CPU / 512Mi memory, requests equal to limits, because the team wanted the strongest eviction protection. Overnight traffic drops and the platform's rightsizer recommends 200m / 256Mi — but the recommendation pipeline patches only requests, leaving limits where they were. That patch would silently convert the Pod from Guaranteed to Burstable, so the API server refuses it outright.

The fix is not to force the patch through; it is to resize requests and limits together so the equality invariant holds, or to accept that the class change requires a replacement Pod created with the new shape from the start.

This is also why a controller that applies VPA-style recommendations in place must do QoS-preserving math on every patch, not just copy numbers over. The GKE workload-resizer project documents exactly this failure: the moment an applyResize path touches only requests, Guaranteed workloads start failing with admission rejections. And the rejection is per-Pod and permanent for that desired state — retrying the same PATCH will never succeed. The only forward path is a recreate: evict the Pod and let the replacement come up in the new class.

One more structural exclusion in the same family: in-place resize is prohibited entirely on nodes using static CPU or Memory Manager policies — typically dedicated, NUMA-pinned training nodes. Those nodes are fully booked by design, so there is nothing to resize into; any autoscaler managing GPU or pinned workloads needs the recreate path as its only path there.

Boundary 2: memory can only shrink to what is actually free​

CPU is compressible — throttle it and the process slows down. Memory is not: hand a container a limit below its current resident usage and the kernel's only recourse is the OOM killer. So the kubelet treats memory-limit decreases as guilty until proven innocent.

Concretely, when a memory shrink arrives with the default NotRequired resize policy, the kubelet performs a best-effort OOM-avoidance check: it compares the proposed new limit against the container's current memory usage, and if usage already exceeds the target, the resize is blocked rather than applied. The resize sits in progress — visible via the PodResizePending and PodResizing conditions that replaced the old free-text resize status in 1.33 — until usage drops below the target or the request is superseded. There is still a small, documented non-zero risk of an OOM-kill in the race between the check and the cgroup write, which is why the check is called best-effort and not a guarantee.

The history here explains the resizePolicy field you will see on every container. In 1.33, the project briefly forbade memory-limit decreases altogether under NotRequired, requiring RestartContainer for any memory shrink; 1.34 relaxed that back into the best-effort check described above. The policy matrix that falls out of it is the thing to internalize:

• CPU-only change with NotRequired: applied in place, no restart, no drama. This is the safe 90 percent of rightsizing.
• Memory change with RestartContainer: the container restarts and the new value applies at start. Predictable, briefly disruptive.
• Memory decrease with NotRequired (the default if you set nothing): applied in place only if current usage fits under the new limit; otherwise deferred, possibly forever if the workload never releases the memory.
• Mixed CPU-plus-memory change: restarts if either resource's policy demands it.

The practical consequence: a downscale recommendation against a container holding 480Mi resident with a proposed 256Mi limit is not a resize at all under the default policy — it is a wish. If your autoscaler treats "PATCH accepted" as "resize done" without watching the actual conditions, it will believe capacity was freed that never was, and your bin-packing math will silently overcommit the node. Watch PodResizePending/PodResizing, and treat a shrink that stays pending past your SLO as a recreate decision, not a patience exercise.

There is a fourth, quieter boundary worth one paragraph because it produces the same symptom: infeasible resizes. If the node cannot accommodate the new values — no headroom left — the resize fails and the only answer is scheduling a replacement Pod elsewhere, which is a scheduler decision, not a kubelet one. In-place resize never moves a Pod between nodes. Any autoscaler built on it still needs the "give up and reschedule" branch for a full node.

The fallback playbook: resize first, recreate on exactly these signals​

Pulling it together, a tenant-facing autoscaler on a self-hosted fleet — Cluster API machines, VPA recommendations, whatever applies the PATCH — needs two code paths and a crisp rule for choosing between them:

Two operational notes complete the picture. First, VPA ships an --in-place-skip-disruption-budget flag (default false) that lets it skip PDB checks specifically for restart-free in-place updates — turn it on only when every container in the Pod carries NotRequired for both CPU and memory (or no policy at all), otherwise the "non-disruptive" update you exempted from the budget can restart containers anyway. Second, in-place resize observability still lags the feature: workload-level agents that cache the cgroup memory limit at startup (the OpenTelemetry collector's memorylimiterprocessor is a documented example) keep enforcing stale thresholds after a resize unless they re-read the cgroup. If your platform injects such agents as sidecars, a resize that the kubelet applied cleanly can still trip a limit the agent never updated — audit your sidecars before you celebrate the feature.

None of this diminishes what 1.36 delivered. Restart-free CPU adjustments and memory increases — the scaling direction that handles traffic spikes, which is when PDBs and in-flight requests matter most — now work at both container and Pod level, on by default. That covers the urgent direction. The boundaries only bite on the way back down and across classes, which is exactly where an autoscaler earns its keep: shrinking costs without killing workloads. Build the recreate path first, keep the resize path for the cases the kubelet will actually accept, and let the two complement each other instead of pretending one replaced the other.

Running tenants on machines you own? Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.