Skip to main content

Kubernetes 1.35 Killed the CPU Restart Tax. Memory Still Pays It.

9 min readDora NodaDora Noda
Share
On this page

For as long as Kubernetes has existed, changing a Pod's CPU or memory allocation meant one thing: kill it and start over. Bump a container from 1 vCPU to 2, and the scheduler tears down the old Pod, finds a new (or the same) node, and boots a fresh container — dropped connections, cold caches, a blip in your uptime graph, all to change two numbers in a spec. Heroku has the same tax: resizing a dyno on the dashboard restarts it, full stop.

That changed on December 17, 2025. Kubernetes v1.35 "Timbernetes" graduated In-Place Pod Resize to General Availability, letting the kubelet rewrite a running container's CPU and memory limits without deleting the Pod. But "no restart" isn't the whole story, and the honest version matters more than the pitch: CPU resizes are genuinely restart-free. Memory resizes, by default, still restart the container. That split — not a blanket "problem solved" — is what a self-hosted PaaS actually has to design its tenant scale-up flow around.

What GA Actually Ships

In-Place Pod Resize has been cooking for a while: alpha in v1.27 (April 2023), beta in v1.33 (April 2025), stable in v1.35. The mechanism is a new resize subresource on Pods — the same pattern Kubernetes already uses for status and scale. Instead of replacing the Pod object, a client patches pods/<name>/resize with new resources.requests/resources.limits, and the kubelet applies them to the already-running container.

A minimal container spec that opts into the new behavior looks like this:

yaml
apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  containers:
  - name: app
    image: myapp:latest
    resources:
      requests: { cpu: "500m", memory: "512Mi" }
      limits: { cpu: "1", memory: "1Gi" }
    resizePolicy:
    - resourceName: cpu
      restartPolicy: NotRequired
    - resourceName: memory
      restartPolicy: RestartContainer

Trigger a live CPU bump against a running Pod with kubectl patch pod web --subresource resize --patch '{"spec":{"containers":[{"name":"app","resources":{"limits":{"cpu":"2"}}}]}}', and the container's restartCount and start time don't move. kubectl describe pod shows the new limit take effect on a process that never stopped running. That's the entire GA payoff in one command.

Three other changes shipped alongside GA that matter for an operator, not just a demo:

  • Memory limit decreases are now allowed. Earlier betas only permitted raising memory limits in-place — lowering one still required a full recreate, because the kubelet couldn't guarantee the container's current usage fit under a smaller ceiling without checking first. GA adds that check.
  • New kubelet metrics and Pod events track resize attempts, so an operator can tell why a resize didn't apply instantly instead of guessing.
  • Deferred resizes retry by priority. If a node doesn't have room for a requested increase right now, the kubelet marks the resize Deferred and retries it as capacity frees up, instead of just failing outright.

It's also worth being precise about what kind of feature this is. Vertical scaling on Kubernetes has existed since 2018 through the Vertical Pod Autoscaler (VPA) — but VPA has always been an external add-on that computes a recommendation and then evicts the Pod to apply it, because there was no lower-level primitive for it to call instead. In-Place Pod Resize is the first time Kubernetes itself has shipped a native mechanism for changing a running Pod's resource allocation — the resize subresource is core API surface, not a controller bolted on top. VPA's newer InPlaceOrRecreate mode (available since VPA 1.2, requiring Kubernetes 1.33+) is best understood as VPA finally getting to call a real primitive instead of working around the absence of one.

Why CPU Is Free and Memory (Usually) Isn't

The resizePolicy field is where the real behavior lives, and it's set per resource, not per Pod. Two values exist:

restartPolicy valueBehaviorTypical resource
NotRequired (default)Apply the new value live, no restartCPU
RestartContainerRestart the container to apply the new valueMemory

The asymmetry isn't arbitrary. CPU is a cgroup-level throttle — cpu.max and cpu.weight are numbers the kernel enforces on a running cgroup, and rewriting them takes effect on the next scheduling tick with zero cooperation from the application. Memory is different: most language runtimes size their heap, GC targets, or connection pools around the limit they saw at startup (a JVM's -Xmx, Node's --max-old-space-size, a database's shared_buffers), and none of them re-read that ceiling mid-flight. So the default combination — NotRequired for CPU, RestartContainer for memory — reflects what's actually safe to do live, not a missing feature.

The composition rule that follows matters for anyone building an automated resize flow on top of this: if a single resize request changes both CPU and memory, and either resource's policy says restart, the whole container restarts. A CPU-only bump on a Pod configured as above is free. A memory-only bump restarts. A combined bump restarts too, even though the CPU half of it didn't need to. If your control plane batches "bump CPU and memory together" into one resize call because it's simpler to implement, you've silently converted every plan upgrade into a restart — including the CPU-only ones a tenant would have gotten for free had you sent it as a separate patch.

The Fine Print Every Fleet Operator Must Design Around

GA doesn't mean unconditional. Four constraints determine whether a given Pod on your fleet gets the in-place path at all:

  • cgroup v2 only. The kubelet rewrites cpu.max and memory.max on a live cgroup without sending the process a signal — that rewrite path doesn't exist under cgroup v1's hierarchy. Any node still running cgroup v1 falls back to the old delete-and-recreate behavior for every resize, silently. This isn't a niche edge case to phase out on your own timeline, either: Kubernetes v1.37 (GA August 26, 2026) drops kubelet support for cgroup v1 entirely and requires containerd 2.0. The ecosystem is converging on cgroup v2 as mandatory within the year regardless of whether you use in-place resize — treat "audit every node for cgroup v2" as one item on your 1.37 upgrade runbook, not a separate project.
  • Static CPU/Memory Manager pods are Infeasible. If a Pod is pinned by the static CPU Manager or static Memory Manager policy (common for latency-sensitive workloads that want exclusive cores), Kubernetes marks its resize request Infeasible rather than attempting it. Those Pods still need the old recreate path.
  • Windows Pods aren't supported. The feature is Linux-cgroup-specific end to end.
  • A full node falls back to Deferred. If the node the Pod is on doesn't have headroom for the new request right now, the resize doesn't fail — it queues and retries by priority as capacity opens up (from bin-packing evictions, other Pods scaling down, etc.). On a fleet that runs nodes at 80%+ utilization as a bin-packing strategy, expect the deferred path to trigger often, not occasionally.

None of these are exotic. A self-hosted PaaS running general-purpose tenant web services on commodity Hetzner nodes with cgroup v2 will mostly avoid the first three — but the fourth, node headroom, is a direct function of how tightly you pack nodes, which is exactly the lever a cost-conscious self-hosted fleet pulls hardest on. A fleet that bin-packs at 80%+ utilization to keep the Hetzner bill down will see the Deferred path trigger routinely; one that runs looser, at 50-60%, will get the instant CPU win almost every time. That trade-off — pack tighter and pay for it in resize latency, or pack looser and pay for it in idle capacity — didn't exist before GA because every resize used to cost a restart regardless of node headroom. Now it's a real dial an operator can tune.

Wiring This Into a Git-Push PaaS's Plan-Upgrade Flow

The obvious use case is the button every PaaS has: a tenant moves their service from a 1 vCPU/1 GiB plan to a 2 vCPU/2 GiB plan. Before v1.35, the only correct implementation was delete-and-recreate — accept the restart, communicate it, move on. After v1.35, a Cluster-API-managed fleet's control plane has a real choice, and the honest design is to make that choice per resource, not per request:

  1. Split the resize into a CPU patch and a memory patch, sent as two separate calls to the resize subresource — not one combined patch. The CPU half applies live under NotRequired; the memory half restarts under RestartContainer. Sending them separately means the tenant's CPU headroom increases immediately even while the memory bump is still queued for its restart window.
  2. Treat Infeasible and Deferred as first-class states in your own API, not just Kubernetes internals to swallow. A tenant-facing "upgrading…" status that's actually sitting in Deferred because the node is full is a materially different problem (rebalance the fleet) from one that's Infeasible because the Pod is CPU-Manager-pinned (recreate is the only path).
  3. Borrow the fallback pattern the Vertical Pod Autoscaler already uses. VPA's InPlaceOrRecreate update mode attempts an in-place resize first and only evicts-and-recreates the Pod if that's not possible — the same two-tier logic a PaaS's own scale-up flow should implement, rather than reinventing it.
  4. Say the true thing to the tenant. "Scaling your CPU: instant, zero downtime. Scaling your memory: applies on the next restart, scheduled for your next deploy or within N minutes" is an honest UX sentence that the mechanics above fully support — and a materially better claim than an unqualified "zero-downtime scaling," which memory-heavy plan bumps will falsify the first time someone tests it.

That's the actual shape of what GA buys a fleet operator: one resource class that's now genuinely free to resize, one that mostly isn't, and a set of edge cases — cgroup version, static pinning, node headroom — that decide which Pods even get to try. Building the tenant-facing feature around that real boundary, instead of the "in-place resize solves scaling" headline, is what keeps the uptime promise honest.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, built on Cluster API so primitives like in-place Pod resize are available to wire into your own scale-up flow instead of hidden behind someone else's dashboard. Star the repo on GitHub or deploy your first app today.

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex