Skip to main content

Kubernetes Swap Went Stable in 1.34: What LimitedSwap Actually Buys a Self-Hosted PaaS on NVMe

9 min readDora NodaDora Noda
Share
On this page

For most of Kubernetes' history, the operating advice on swap was blunt: turn it off. The kubelet refused to even start on a swap-enabled node unless you passed --fail-swap-on=false, and the reasoning was sound at the time — a scheduler that can't see swapped memory produces wildly unpredictable pod behavior, and swapping to a spinning disk turns a memory spike into a multi-second stall. That changed for real in Kubernetes v1.34, released in August 2025: node swap support graduated from beta to stable, and swapBehavior: LimitedSwap is now a production-supported per-node tuning knob, not an experimental flag.

The headline number is simple: size a Hetzner NVMe swap partition at roughly 25% of a node's RAM, and a self-hosted PaaS bin-packing bursty tenant workloads gets real burst headroom on top of its existing request-based packing — without provisioning full peak RAM for every tenant up front. That's a genuine lever. It's also a narrower one than "pack more tenants" makes it sound, and the rest of this post is about exactly where the edges are.

What LimitedSwap Actually Does

The kubelet's memorySwap.swapBehavior field has two settings. NoSwap is the default and unchanged from before: kubelet itself can run on a swap-enabled node, but pods are barred from touching swap at all. LimitedSwap is the new stable behavior, and it only works on cgroup v2 — cgroup v1 hosts don't get swap support at all, full stop.

Under LimitedSwap, the kubelet computes a per-container memory.swap.max using one formula:

text
swap_limit = (container_memory_request / node_memory_capacity) × node_swap_size

That formula only produces a nonzero number for one QoS class:

QoS classmemory.swap.maxWhy
Guaranteed (request == limit)0The formula's numerator is defined as zero for these pods by design — full swap isolation
BestEffort (no request set)0No memory request means the formula's numerator is zero
Burstable (request < limit)Proportional to requestThe only QoS class that gets swap access at all

Kubernetes also recommends keeping system.slice — the kubelet itself, container runtime, and other node daemons — pinned to memory.swap.max=0, so a noisy tenant workload can never swap out the processes actually running the node.

The practical upshot: enabling LimitedSwap on a node doesn't change anything for your Guaranteed-QoS control-plane components or your BestEffort background jobs. It only opens a proportional, request-sized swap allowance for Burstable pods — which is exactly the QoS class a bursty, rarely-fully-active tenant workload runs under.

What It Protects Against — and What It Still Won't Save You From

This is the part worth being precise about, because "swap prevents OOM kills" is only true for one specific failure mode. The clearest public numbers on this come from ScaleOps' testing of swap behavior under GPU-inference workloads — worth flagging up front that these are AI-inference benchmarks, not a general web-service test. The exact percentages below are workload-specific; the mechanism they demonstrate is not — it's kernel and cgroup behavior that applies to any Burstable pod, GPU or not.

Four scenarios from that testing, all on a container with a 512Mi memory ceiling:

ScenarioSwap configOutcome
Brief burst above steady state, UnlimitedSwapNo proportional capPod survived; GPU utilization dropped from 94% to 73.5% (a 26% hit) while ~1.6GB paged out
Same burst, undersized LimitedSwap entitlement (~196MB)512Mi limitPod was OOMKilled anyway, despite 145,000 page-out operations — the entitlement was too small to cover the gap
Same ~196MB entitlement, but a correctly-sized 2Gi limit2Gi limitPod survived with zero errors — the cgroup limit, not the swap, was doing the real work
Sustained memory growth, not a transient burstSwap enabledLatency degraded 8.19x (14.2s → 116.5s p50) before the process was still eventually killed

Read across those four rows and the pattern is consistent: swap absorbs a transient spike — a request handler that briefly allocates above its steady-state footprint and then gives the memory back. It does not turn an undersized limit into a correctly-sized one, and it does not save a workload whose memory usage is genuinely, persistently growing. That workload still gets killed. Swap just makes the failure slower and messier — hundreds of thousands of page operations and a latency cliff — instead of an instant, clean OOM kill. If your mental model of LimitedSwap is "now I can undersize memory limits and let swap cover the difference," row two is the reason that model is wrong.

Why NVMe Changes the Math

The old "never enable swap on Kubernetes" folklore was formed in an era when swap almost always meant a spinning disk, and the latency gap between RAM and rotational storage is enormous — random 4K reads on an HDD run 5–10 milliseconds, five to six orders of magnitude slower than RAM. A page fault that hits swap on spinning disk doesn't just slow a request down, it can stall the whole request pipeline long enough to trip liveness probes.

NVMe closes most of that gap. A random 4K read on a modern NVMe device runs 20–70 microseconds — roughly 100x faster than a SATA SSD's 100–200 microseconds, and multiple orders of magnitude faster than spinning disk. That's the reason LimitedSwap going stable in 2025 is a materially different proposition than swap-on-Kubernetes would have been a decade ago: the same mechanism, running on the local NVMe every Hetzner Cloud server ships with by default, turns a page fault into a latency tax measured in tens of microseconds instead of a multi-millisecond stall.

It's still not free — the 8.19x latency degradation number above happened on NVMe, not spinning disk, and swap I/O still competes with the workload's own disk activity. NVMe makes the trade survivable at scale; it doesn't make swap invisible.

What This Buys in Node Density

Here's the part that's specific to a self-hosted PaaS's own bin-packing math, using a generic web-tenant profile rather than the GPU-inference numbers above — a small Node/Rails-style API service, not a model-serving workload. Take a Hetzner CX43 (8 vCPU / 16 GB RAM, roughly $18.35/month post the June 2026 price adjustment) and size swap at the commonly recommended 25% of node RAM: a 4GB NVMe partition.

The comparison across three representative Burstable tenant sizes:

Tenant size (request/limit)Tenants/node, request-only packingTenants/node, request packing + 4GB swap headroomCost per tenant/mo (with swap)
Small (256Mi / 512Mi)62 (16GB ÷ 256Mi)Same 62 by request, but each now honors its full 512Mi limit during a burst instead of being capped at 256Mi or evicted$0.30
Medium (512Mi / 1Gi)31Same 31 by request, full 1Gi limit now honorable under burst$0.59
Large (1Gi / 2Gi)15Same 15 by request, full 2Gi limit now honorable under burst$1.22

That table is worth reading carefully, because swap doesn't change the count in either column — the request-based packing math (how many tenants fit by their baseline memory request) is unchanged either way. What swap changes is whether the limit — the burst ceiling every tenant was promised — is actually something the node can honor without either evicting a bursting tenant or having provisioned full peak RAM for all 15–62 tenants simultaneously, which no realistic bursty workload needs at the same instant. Without swap, honoring every tenant's limit means either buying that peak RAM upfront (a bigger, pricier node) or accepting that a burst evicts someone. With swap, the node absorbs some number of simultaneous bursts by paying a latency tax on local NVMe instead of paying for unused RAM the other 51 weeks of the month.

Why a Managed PaaS Has No Equivalent Lever

None of this is available to a team running on Heroku, Railway, or Render, and it's not an oversight — it's structural. Heroku's dynos are fixed-size and isolated per app: a Standard-1X dyno gets its 512MB and nothing more, and exceeding it trips an R14 (or R15, if it's a slow leak) memory-quota error and a dyno restart. There's no per-tenant swap knob to turn, because there's no shared node whose swap Heroku could safely expose to one customer's dyno without also exposing it — and its noisy-neighbor risk — to every other tenant sharing that physical host. Railway and Render's fixed-tier instance model has the same shape: the instance boundary the customer buys is also the isolation boundary the vendor sells, and blurring it to let one tenant borrow another's headroom is a support and trust problem no managed platform wants to take on for a cost optimization its customers won't even see credited to their bill.

That's the actual reason "own the node" keeps showing up as a structural advantage on a self-hosted platform, not just a pricing one. A vendor renting you an isolated instance has every incentive to keep swap, overcommit, and bin-packing decisions on their side of the fence — because those decisions only pay off across a shared pool of tenants, and a rented instance isn't shared, it's sold. An operator who owns the physical Hetzner box is the only party positioned to actually make the swap-for-density trade, because they're the only party who captures both sides of it: the saved RAM and the tail-latency cost.

The Practical Version

Enable LimitedSwap for Burstable-QoS tenant workloads only — never for Guaranteed pods, which get zero swap by the formula regardless, and don't rely on it for BestEffort jobs, which also get zero. Size the swap partition at roughly 25% of node RAM on local NVMe, not spinning storage. Treat it as burst insurance that lets a node honor tenant memory limits without provisioning peak RAM for every tenant at once — not as a substitute for correctly sizing memory requests in the first place. Row two of the ScaleOps table is the cautionary tale: an undersized limit still gets killed, swap or not.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Node-level tuning like this is exactly the kind of lever a Cluster-API-managed fleet on owned Hetzner hardware gets to pull that a rented instance never will. Star the repo on GitHub or deploy your first app today.

Sources

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex