All or Nothing: What Kubernetes v1.36's PodGroup Scheduling Buys a Self-Hosted PaaS That Packs Tenants Tight

Every self-hosted PaaS eventually meets the same failure mode, and it never looks like a scheduler bug. A tenant's four-pod service lands three pods on a Friday afternoon. The fourth stays Pending all weekend — not enough room on any one node — while the three running pods burn CPU serving nothing, because the service only works when all four are up. On Monday the platform team adds a node, the fourth pod lands, and everyone files it under "we needed more capacity." They usually didn't. They needed the scheduler to treat the four pods as one decision.

Here is the verdict up front: Kubernetes v1.36 ships the native primitive that fixes this — a dedicated PodGroup scheduling cycle with all-or-nothing gang placement, plus first-iteration rack-aware co-location and group-aware preemption. On a tightly packed fleet of fixed, owned nodes it eliminates an entire class of stranded-capacity waste. But every word after "ships" needs an asterisk: the APIs are alpha and off by default, the v1alpha1 API from 1.35 was replaced wholesale by v1alpha2, heterogeneous groups still aren't guaranteed a placement, and topology constraints cannot trigger preemption yet. Test-cluster material, not production material. The rest of this post is the evidence, starting with the money.

The partial-placement tax on a three-node mini-fleet​

Take a small, representative fleet: three fixed 8-vCPU / 32 GB nodes — hardware you own, so there is no cluster-autoscaler fairy to summon a fourth box. Two tenants share it:

• Tenant A runs a four-pod stateful web tier, 2 vCPU per pod (8 vCPU total). It only serves traffic when all four pods are up.
• Tenant B runs six small single-pod workers, 1 vCPU each (6 vCPU total). Each worker is useful alone.

Late in the day the fleet is fragmented: node 1 has 3 vCPU free, node 2 has 3 free, node 3 has 1 free. Seven vCPU free in total — one short of what tenant A needs whole.

The default scheduler thinks pod by pod. It binds two of A's pods (one onto node 1, one onto node 2, 4 vCPU committed), leaves the other two Pending, and the two running pods serve zero requests. Meanwhile B's workers, which could have productively used 6 of those 7 free vCPU, queue behind A's stranded reservations. The waste is not just 4 vCPU of useless running pods — it is the head-of-line blocking, the retry churn in the scheduling queue, and the descheduler or human that later has to unpick a placement that should never have happened.

With a minCount: 4 gang policy, the same moment plays out differently: the scheduler evaluates A's four pods as one atomic unit, finds the fleet one vCPU short, binds none of them, and returns the whole group to the queue for a backoff retry. All 7 free vCPU stay available for B's workers, and A lands whole the moment 8 vCPU open up. Nothing half-runs. Nobody pages.

What that does to per-tenant unit cost​

On owned hardware the cost side is fixed, so packing efficiency is the entire margin story. Take the mini-fleet at an illustrative €100 per box per month — €300 fixed whether it serves four tenants or ten. The fleet has 24 vCPU of raw capacity:

Same machines, same power bill, ~30% lower unit cost — purely from landing multi-pod services whole instead of stranding fragments. These are illustrative numbers, but the shape survives contact with real fleets: when capacity is fixed, every vCPU stranded behind a half-placed service is margin you paid for and cannot sell. That is the number the title promises, and it is why a PaaS operator should care about a scheduler feature even before it is production-ready.

What v1.36 actually shipped​

The headline from the May 2026 Kubernetes blog post is an architectural split. In v1.35, pod-group membership and runtime state were embedded in the Workload object; in v1.36 the concerns separate cleanly: the Workload API is a static template, and the new PodGroup API is the runtime object the scheduler actually reads. Both live in scheduling.k8s.io/v1alpha2, which completely replaces the v1alpha1 API — an upgrade that rewrites your manifests, not just your feature gates. Because the scheduler reads the PodGroup directly without watching or parsing the Workload, status updates can shard per replica instead of funneling through one object — the scalability win that makes group scheduling viable past toy clusters.

The minimal shape is three objects. A controller (the Job controller has first-phase integration already) stamps a template:

apiVersion: scheduling.k8s.io/v1alpha2
kind: Workload
metadata:
  name: training-job-workload
  namespace: some-ns
spec:
  podGroupTemplates:
  - name: workers
    schedulingPolicy:
      gang:
        minCount: 4

The controller stamps out a runtime PodGroup carrying the live policy and status:

apiVersion: scheduling.k8s.io/v1alpha2
kind: PodGroup
metadata:
  name: training-job-workers-pg
  namespace: some-ns
spec:
  podGroupTemplateRef:
    workload:
      workloadName: training-job-workload
      podGroupTemplateName: workers
  schedulingPolicy:
    gang:
      minCount: 4
status:
  conditions:
  - type: PodGroupScheduled
    status: "True"

And each pod points at the runtime group — note the field rename, workloadRef from 1.35 is gone:

apiVersion: v1
kind: Pod
metadata:
  name: worker-0
  namespace: some-ns
  schedulingGroup:
    podGroupName: training-job-workers-pg

The engine behind it is the new PodGroup scheduling cycle. When the scheduler pops any group member off the queue, it pulls the rest of the group's queued pods, sorts them deterministically, takes a single snapshot of cluster state, and places the whole group atomically: all schedulable members bind together, the rest wait; if the group can't meet minCount, nothing binds and everything retries after backoff. Two details matter operationally. First, pods already bound stay bound — a later cycle never unassigns running pods, so a group that grows after partial scheduling doesn't thrash. Second, the scheduler holds pods in PreEnqueue until minCount is satisfiable, which is what keeps a hungry group from clogging the active queue.

Rack-aware placement without affinity spaghetti​

Gang placement answers "do we all land"; topology-aware scheduling answers "do we land together." For anything latency-sensitive — a tenant's cache plus app tier, an inference worker plus its sidecar — scattering pods across racks trades placement success for tail latency. Today operators express that with layered podAffinity and topologySpreadConstraints, which interact in ways nobody can predict without a whiteboard.

v1.36 lets the group declare the constraint directly:

apiVersion: scheduling.k8s.io/v1alpha2
kind: PodGroup
metadata:
  name: topology-aware-workers-pg
spec:
  schedulingPolicy:
    gang:
      minCount: 4
  schedulingConstraints:
    topology:
    - key: topology.kubernetes.io/rack

Under the hood the scheduler generates candidate node subsets matching the constraint (a new PlacementGenerate extension point), verifies each fits the whole group, and scores them (PlacementScore) by fit and utilization. In plain English: it shops for the best rack-shaped hole instead of placing pods one by one and hoping they cluster.

How it differs from the old tools, in three rows: podAffinity says "this pod likes pods like that" (pairwise, pod-centric); topologySpreadConstraints says "spread evenly across zones" (statistical, best-effort); the PodGroup topology constraint says "this group lands inside one rack or it doesn't land" (atomic, group-centric). The third is the only one a bin-packing PaaS can reason about as a unit-cost input — co-location decided correctly the first time instead of corrected by a descheduler after the fact.

Preemption that thinks in groups​

The quiet companion feature is workload-aware preemption, and it fixes a real gap in the default preemptor: today preemption is evaluated per node, so a high-priority group needing room across two nodes can fail even when evicting one pod from each would free enough in aggregate. The new mechanism treats the PodGroup as a single preemptor unit and searches cluster-wide, evicting from multiple nodes at once to make the whole group fit.

Two new PodGroup fields drive it — a group-level priority overriding individual pod priorities, and a disruptionMode controlling whether the group's own pods can be picked off individually or only evicted all together:

apiVersion: scheduling.k8s.io/v1alpha2
kind: PodGroup
metadata:
  name: victim-pg
spec:
  priorityClassName: high-priority
  priority: 1000
  disruptionMode: PodGroup

In 1.36 these fields are honored only by the workload-aware preemption path — the default pod-by-pod preemptor ignores them, with wider support slated for later releases. Combined with the topology work, the direction is clear (group-scoped priority plus group-scoped placement), but the two halves don't fully meet yet: topology-aware scheduling will not itself trigger preemption to satisfy a constraint. A group that needs both "together in one rack" and "evict someone to fit" still waits.

The honest limits checklist​

This is the section the title's second half demands — what PodGroup in 1.36 is not versus a custom scheduler you build yourself:

• Alpha, off by default, and a breaking API migration. You enable GenericWorkload / GangScheduling gates plus the v1alpha2 API group, and anything written against v1alpha1 gets rewritten. The beta graduation is on the stated roadmap, not in your cluster.
• Homogeneous groups get the guarantee; mixed groups don't. If every pod in the group has identical scheduling requirements, the algorithm is expected to find a placement when one exists. Heterogeneous groups, groups with inter-pod dependencies, and groups with intra-group affinity can all fail to place even when a valid placement exists — deterministic processing order is the documented culprit for the last one. A PaaS serving arbitrary tenant shapes lives exactly in the un-guaranteed region.
• No topology-triggered preemption yet, as above. Rack-aware placement that can't evict is a polite request under contention, not a guarantee.
• No queues, no fair-share, no quota borrowing. Native gang scheduling places one group atomically; it does not decide whose group goes first across tenants. That is still Volcano / Kueue territory — they add the multi-queue management, fair-sharing, and quota borrowing the in-tree feature deliberately doesn't, which is why both projects are integrating with the Workload API rather than being replaced by it.

The decision rule: enable the native feature to learn it, keep Volcano or Kueue where tenant fairness matters, and only consider a custom scheduler if you need placement guarantees for heterogeneous groups that 1.36 explicitly disclaims. Almost nobody needs the third option; many teams convince themselves they do before trying the first.

What to do Monday morning​

1. Stand up a throwaway 1.36 test cluster with GenericWorkload, GangScheduling, and scheduling.k8s.io/v1alpha2 enabled — never your fleet.
2. Recreate the mini-fleet above (3 nodes, one 4-pod gang plus single-pod backfill) and watch the scheduling queue: confirm all-or-nothing under contention and clean backoff retries.
3. Add the rack topology constraint with labeled fake-rack nodes and verify co-located picks — then verify the failure mode: contend the rack and confirm it waits instead of preempting.
4. Exercise workload-aware preemption across two nodes with disruptionMode: PodGroup and confirm the all-or-nothing eviction semantics.
5. Measure what matters for your unit-cost model: stranded vCPU before/after on your own tenant mix, and queue latency for gangs under contention. Those two numbers decide your 1.37 upgrade priority when beta lands.

Kubernetes spent a decade scheduling one pod at a time and bolting group semantics on through external schedulers. v1.36 is the release where the default scheduler starts thinking in workloads — atomically, topology-aware, and preempting as a unit. For a self-hosted PaaS whose margin is packing density on fixed hardware, that is the most financially relevant scheduler work in years. Just let it bake until beta before it touches a tenant.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.