Kubernetes 1.37's DRA Bridge Is GA: Migrate Your GPU Pool Without Rewriting a Single Tenant Manifest

Every GPU migration you have ever postponed had the same blocker, and it was never the driver. It was the manifests. Hundreds of tenant pod specs saying nvidia.com/gpu: 1, every one of them written against the old device-plugin API — and every DRA migration guide until now effectively told you to touch all of them, because DRA-only nodes advertise no nvidia.com/gpu in node.status.allocatable at all. Adopt the new scheduler, rewrite the fleet. That was the deal.

Kubernetes 1.37 "Garhwal" tears up that deal. DRA Extended Resource support (KEP-5004) graduated to stable in the September 3 DRA-updates post: a DRA driver can now satisfy requests made through the traditional extended-resource API with no ResourceClaim on the workload and no device plugin beside the driver. Your tenants keep writing nvidia.com/gpu: 1. The allocation underneath moves to DRA. Here is the one-screen map — what is GA now, what stays gated, and the single constraint that decides your rollout order — followed by the playbook for each row.

What "stable" actually means this time​

The headline feature has been on a conspicuously steady path: KEP accepted in 1.34, alpha in 1.35, beta in 1.36, stable in 1.37 — what the release post calls "a milestone the team has been building toward for three straight releases." The mechanism is small enough to state completely. A DeviceClass declares an extended resource name, and pods requesting that name get matched to a device through DRA. The scheduler synthesizes a special ResourceClaim behind the scenes, so the pod spec carries no claim, no template, no new API object. Conceptually, the entire migration looks like this:

apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: gpu-class
spec:
  extendedResourceName: nvidia.com/gpu

while every tenant pod keeps saying what it has always said:

resources:
  limits:
    nvidia.com/gpu: 1

Three sibling graduations make the bridge operationally complete rather than merely compatible. ResourceClaim status with standardized network data (KEP-4817) adds a devices field to claim status, letting drivers report per-device state — interface name, MAC address, and assigned IPs for network devices. Before this, once a device was configured into a pod, nothing else in the system could learn the IP it got; now a controller can build network services on reported device addresses instead of scraping them out of band. Device taints and tolerations (KEP-5055) push the familiar node-taint model down to individual devices: a driver marks a GPU degraded, or an admin applies a DeviceTaintRule cluster-wide without reconfiguring any driver, and the scheduler skips that device for new pods. Pods already holding a tainted device are evicted automatically unless their claim tolerates the taint. And the standardized resource.kubernetes.io/numaNode attribute lands directly as stable — a naming registration with no feature gate — so devices from different vendors can finally be compared on the same NUMA node instead of each driver inventing its own label for it.

For a self-hosted fleet, the taints feature deserves a second look. Until now, one sick GPU meant cordoning or draining the whole node: CPU, memory, and seven healthy accelerators held hostage by one bad device. Per-device cordoning turns a node evacuation into a single-device quarantine. On a four-GPU Hetzner box, that is the difference between losing 25% of a node and losing 100% of it during the hours — sometimes days — you wait on a hardware verdict.

The migration playbook: pool by pool, manifests untouched​

The rollout that the GA bridge enables has three phases, and the middle one contains the only hard constraint in this post — so read that part twice.

Phase 1: coexistence on separate pools. Install the DRA driver (NVIDIA's documents DRAExtendedResource as enabled by default since 1.36) alongside your existing device plugin, but on different node pools. Old pools keep serving nvidia.com/gpu through the plugin; new or drained pools serve the same resource name through DRA. Tenants change nothing. Your scheduler sees one resource name with two backends behind it, partitioned by node.

Phase 2: drain and flip, one pool at a time. This is where the constraint bites: the same extended-resource name cannot be backed by the device plugin and DRA on the same node. A node advertises nvidia.com/gpu allocatable entries from the plugin, or it serves the name through DRA — never both. So migration is a per-node-pool operation: cordon a pool, drain it, remove the plugin, enable the DRA driver with the DeviceClass mapping, uncordon. Validate with a canary pod requesting one GPU before the tenants' real workloads land. Repeat per pool until the plugin DaemonSet has no nodes left to serve and can be deleted.

Phase 3: adopt the GA sidecars. Once a pool is DRA-backed, wire the two stable companions in: a DeviceTaintRule policy for your known-bad-device workflow (firmware mismatch, ECC errors, the GPU that runs 15 degrees hotter than its siblings), and claim-status consumption wherever you currently scrape device addresses by hand. Standardize any NUMA-aware placement on resource.kubernetes.io/numaNode while you are touching those constraints anyway.

Note what this playbook deliberately does not include: rewriting tenant manifests, dual-running both backends on one node, or flipping any beta gate to get there. If a migration plan asks tenants to change pod specs, it is a pre-1.37 plan wearing new clothes.

Where to stop: the two gates that stay off​

Two features sit exactly one step beyond "ready," and the correct move for both is the same: know what they unlock, leave them disabled, calendar a revisit for 1.38.

DRAWorkloadResourceClaims (KEP-5729, beta, off by default). Today a claim is reserved per pod, with sharing capped at 256 pods through the per-pod reservation limit. The beta lets Workload and PodGroup objects reference ResourceClaims directly, so one claim spans an entire gang-scheduled group — the natural companion to the workload-aware scheduling (gang scheduling, PodGroups) that also went beta in 1.37. Enabling it means flipping the gate on four components at once — API server, controller manager, scheduler, and kubelet — which is precisely the blast radius a small ops team should not accept for a beta during a driver migration. Revisit when it goes GA or when a real training job outgrows the 256-pod reservation cap, whichever comes second.

DRADeviceCompatibilityGroups (KEP-5963, alpha, off by default). Drivers tag partitions of a device — MIG versus vGPU profiles on the same physical GPU — with compatibility groups so the scheduler rejects impossible combinations up front instead of the driver failing at node-prepare time. Valuable the moment you slice GPUs; premature while your pools are whole-GPU. The rule of thumb: enable it in the same quarter you start partitioning, not before.

A third item belongs on the watch list rather than the stop list. Fractional capacity ranges (KEP-5075, beta behind DRAFractionalCapacityRange) now allow fractional values in CapacityRequestPolicyRange, which is the upstream primitive that fractional-GPU sharing will eventually be expressed in. "Eventually" is doing real work in that sentence — see the next section.

The scheduler win hiding at the bottom of the changelog​

Buried in the alpha list is the performance item with the widest blast radius: the PreQueueingHint extension point (KEP-6132). Every ResourceClaim event used to trigger a full scan of every unschedulable pod — O(N²) work during a large scale-up, exactly when the scheduler is already hottest. The DRA plugin now uses a pod-informer index to narrow the requeue to the pods actually affected: O(1), and roughly double the scheduling throughput in early benchmarks.

Why should a fleet of a dozen GPU nodes care about scheduler throughput? Because GPU scale-ups are the worst case for this pathology. A tenant submits fifty inference replicas at once; forty-eight go unschedulable waiting on one claim event; each event re-scans all forty-eight. On a small fleet the absolute numbers are modest, but the shape is identical — and it compounds with cluster-autoscaler latency, turning "two minutes to scale" into "nine minutes, then everything lands at once." The gate (SchedulerPreQueueingHints) is alpha, so this is not a 1.37 action item. It is the reason your 1.38 upgrade notes should have a scheduling-benchmark line: record time-to-place for a fifty-pod GPU burst now, compare after the gate goes beta, and you will know — rather than guess — what the fix bought you.

What this means if you share GPUs with HAMi​

Many self-hosted GPU pools do not serve whole devices at all. HAMi — the heterogeneous-sharing scheduler, now a CNCF Incubating project — virtualizes GPUs into fractional slices with memory and core enforcement, and it ships its own k8s-dra-driver for DRA-native partitioning. Where does the 1.37 bridge leave those pools?

Be precise about the layering, because two different "sharing" stories are converging and they are not the same thing. HAMi's vGPU model enforces fractional shares today through its device plugin, scheduler extender, and runtime hooks. DRA's fractional capacity ranges are the upstream API such shares will eventually be requested through. The 1.37 playbook for a HAMi pool is therefore: migrate the whole-GPU request path onto the GA bridge first (most HAMi deployments still serve some whole-GPU tenants), pilot fractional capacity ranges beside — not instead of — HAMi enforcement, and treat DRA-native partitioning as the successor to evaluate per HAMi release, not per Kubernetes release. Nothing in 1.37 retires your vGPU scheduler. What 1.37 does is make sure that when the DRA-native path is ready, the manifests on top of it will not need to change — because the extended-resource spelling those manifests already use is now a first-class DRA citizen.

The shape of the next year​

Step back and the pattern across 1.35 through 1.37 is unmistakable: alpha in one release, beta in the next, stable in the one after, each rung announced with migration tooling rather than just new API. Extended resources rode exactly that escalator. Workload-scoped claims and compatibility groups are on it now, which makes their GA in 1.38 or 1.39 the base case, not a hope. A platform team planning GPU capacity on owned hardware can read this cadence as a schedule: whole-GPU DRA migration this quarter on stable APIs, shared-claim gang scheduling next, partitioned-device topology after that — each step landing on the stable rung of the last.

If you run your own fleet, the action list fits on an index card: pick your emptiest GPU pool, install the DRA driver with a DeviceClass carrying your existing resource name, flip one pool, and watch tenant pods schedule with specs they wrote years ago. The migration you postponed because of the manifests is now a node operation, not a fleet-wide code change. That was always the right shape for it.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.