CNCF Says Cloud Native Already Runs AI. Here's What Actually Transfers to a Git-Push PaaS

CNCF's March 2026 report, "The Platform Under the Model," makes a specific claim: production AI engineering — model serving, agent orchestration, GPU scheduling — is converging on the same cloud-native primitives (Kubernetes, containers, declarative APIs) that already run ordinary web workloads, rather than spawning a parallel ML-specific stack. For a CPU-first, git-push PaaS built on Cluster API that has so far treated GPU inference as a bolt-on, that claim is either great news or a trap, depending on whether it's actually true yet.

It's both — split cleanly down the middle. At the control-plane layer, the claim holds up: containers, declarative CRDs, GitOps deploys, and OpenTelemetry observability really are the same primitives whether the pod runs a Node.js server or an inference engine. At the scheduling and serving layer, it doesn't hold up yet: GPU-aware scheduling only reached general availability in April 2026, disaggregated inference needs a purpose-built control plane that didn't exist a year ago, and CNCF itself just spent a year building an entire additional certification layer because base Kubernetes conformance wasn't sufficient proof a platform could actually run AI workloads.

Here's what that table means concretely for a fleet that's spent the last year assuming "we already run Kubernetes" was most of the answer.

What genuinely is shared today​

The control-plane argument is the strongest part of CNCF's case, and it's not just rhetoric. The primitives a git-push PaaS already leans on for ordinary web apps really do carry over to AI workloads without modification:

• Containers and declarative CRDs. An inference server packages the same way an Express app does — image, entrypoint, resource requests. Kubeflow, the project CNCF points to as evidence of convergence, has grown into a top-30 CNCF project specifically by expressing pipeline orchestration and model serving as CRDs rather than a bespoke ML control plane.
• GitOps deploy. kubectl apply doesn't care whether the manifest describes a stateless web service or a LLMInferenceService. The deploy mechanism — commit, reconcile, roll out — is identical.
• Policy and identity. Open Policy Agent and SPIFFE/SPIRE provide the same governance primitives for an inference workload's access to a model-weights bucket as they do for a web app's database credentials.
• Observability, extended. Prometheus and OpenTelemetry remain the collection layer; AI workloads just add new metric types on top — tokens per second, time-to-first-token, queue depth, KV-cache hit rate. The pipe is the same, only the payload changed shape.

Adoption numbers back up that the industry is using Kubernetes as the substrate: 82% of container users run Kubernetes in production, and 66% of organizations use Kubernetes specifically to host generative-AI workloads. Those are strong numbers for "the infrastructure converged."

Where the picture gets murkier is a separate, oft-cited CNCF/SlashData stat: only 41% of professional AI developers currently identify as "cloud native." That's not a before/after measurement of the same group — it's two different populations (organizations running infra vs. individual AI practitioners describing their own identity) — so it's not proof the convergence stalled. But it is a real signal worth sitting with: even where the infrastructure has converged, a large share of the people building on top of it don't yet think in cloud-native terms. That's a knowledge gap a platform's own tooling and docs have to bridge, not something the shared primitives fix by existing.

What's genuinely different: GPU scheduling​

This is the part of the claim that's aspirational rather than operational, and the timeline makes the case on its own. Dynamic Resource Allocation (DRA) — the Kubernetes API that lets the scheduler actually understand heterogeneous accelerators (GPU type, memory hierarchy, topology, time-sharing) instead of treating a GPU as an opaque integer to count — only reached general availability in Kubernetes v1.36, shipped April 22, 2026. Before that, GPU scheduling on Kubernetes ran through the legacy Device Plugin API, which can allocate whole devices but can't express partial sharing, time-slicing, or post-start allocation.

The ecosystem has moved fast since GA: NVIDIA donated its own DRA driver to CNCF at KubeCon Europe 2026, and the KAI Scheduler — previously an NVIDIA-governed batch scheduler — was accepted as a CNCF Sandbox project the same event. That's real, fast convergence. But "fast" is doing work in that sentence: a Cluster-API fleet that assumed GPU scheduling was a solved, mature Kubernetes primitive would have been wrong for the fleet's entire operating history up to three months before this article, and is only now reasoning about a GA'd — not battle-hardened — API.

Serving compounds the gap. A stateless web app scales by starting more identical, cheap-to-boot containers. An inference workload doesn't:

• Container image pull: a 15GB inference image takes 4–8 minutes to pull, versus seconds for a typical Node.js image.
• Model weight transfer: loading a 70B-parameter model at FP16 from storage into GPU VRAM takes 40+ seconds on its own.
• CUDA context init and graph capture: another 10–30 seconds before the first token can even begin generating.

Put together, a cold-started inference pod can take 30–90 seconds or more before serving its first request — smaller models land at the low end of that range, larger ones push well past it, so the honest framing is a range driven by model size, not a single fixed number. Either way, none of it maps onto a Deployment scaling from zero the way a web app does; it's the reason llm-d — a distributed-inference control plane that entered the CNCF Sandbox in March 2026 — exists at all, splitting prefill and decode across pods and moving KV cache between them so a fleet doesn't eat that cold-start cost on every scale event. KServe wraps that into an LLMInferenceService object with the same declarative shape as a Deployment, but underneath it is doing meaningfully more work than a Deployment ever had to.

What's genuinely different: agent orchestration​

CNCF's claim names three convergence axes — model serving, GPU scheduling, and agent orchestration — and the third gets the least airtime, which is worth correcting, because it splits the same way serving does: cleanly convergent at the unit-of-work layer, still bespoke one layer up.

Agents-as-pods is real and works today. Kagent, a CNCF Sandbox project, defines agents as CRDs — an agent is versioned, rolled out via kubectl and GitOps, and runs as a pod like anything else. The newer Agent Sandbox project under Kubernetes SIG Apps takes the same approach for the execution environment itself: a SandboxClaim against a SandboxTemplate hands an agent a pre-warmed, isolated environment without a bespoke sandboxing layer sitting outside Kubernetes. Neither of these required inventing new infrastructure primitives — they're CRDs and controllers, full stop.

Multi-agent orchestration is not there yet. Sequencing a multi-step agent workflow, having agents discover each other, and handling failure across that workflow currently has no built-in workflow engine or task-graph primitive in Kubernetes itself — that logic still lives in the agent framework sitting on top (LangGraph, CrewAI, or a platform's own glue code), not in a shared cloud-native object every platform can point at. AIBrix's answer, notably, is a hybrid: Kubernetes for coarse-grained resource management, Ray layered on top for the fine-grained orchestration Kubernetes doesn't natively express. That's the same pattern as serving — Kubernetes provides the substrate, but the actual orchestration logic is a separate, still-maturing layer bolted on rather than absorbed.

The conformance program is the tell​

If cloud-native primitives already fully served AI workloads, there'd be nothing left to certify beyond standard Kubernetes conformance. CNCF's own actions say otherwise. The Certified Kubernetes AI Conformance program, launched in November 2025, explicitly requires a platform to first pass standard Kubernetes conformance and then clear a separate, additional bar: accelerator support, AI-aware networking, distributed scheduling, AI-specific observability, and — as of the 2026 update (v1.35 "KARs") — hardware orchestration, agentic-workload validation, and in-place pod resizing.

Certified platforms nearly doubled from 18 to 31 between late 2025 and March 2026, which is a genuinely fast adoption curve. But the existence of the program is itself the evidence: CNCF built a whole second checklist because passing the first one — the "same primitives" checklist — wasn't sufficient proof a platform could actually run AI workloads reliably. That's not a knock on the convergence thesis; it's the honest shape of where the thesis currently stands.

What this means for a git-push PaaS's roadmap​

For a fleet running on Cluster API against owned Hetzner hardware, the actionable read isn't "wait for AI primitives to mature" or "build everything bespoke" — it's to sort the roadmap by which side of the table above a piece of work falls on:

• Lean on what's already shared. The declarative-deploy path, GitOps rollout, and OTel pipeline a git-push platform already built for web apps don't need a parallel AI-specific version. Extending them with new metric types (TTFT, tokens/sec) is additive work on an existing pipe, not a new one.
• Treat GPU scheduling as new infrastructure, not a config flag. A DRA-based node pool, added via Cluster API Provider Hetzner, is genuinely new work — it's a capability that's existed in GA form for barely three months. Budget it like new infrastructure, because it is.
• Adopt llm-d/KServe rather than hand-roll serving. The cold-start math above is exactly the problem that control plane exists to solve; building a bespoke scale-from-zero path for inference pods means re-solving a problem the ecosystem already has a maturing answer for.
• Keep agent execution on cloud-native primitives, keep orchestration logic separate. Running agents as CRDs/pods (the Kagent/Agent Sandbox pattern) is safe to build on now. Multi-agent workflow sequencing is not a primitive to wait for — it's still framework-level logic a platform has to own or integrate, the same way AIBrix pairs Kubernetes with Ray rather than expecting Kubernetes alone to do both jobs.
• Read the AI Conformance checklist as a literal punch list, not a marketing badge — it's CNCF's own enumeration of the gap between "runs on Kubernetes" and "runs AI workloads well on Kubernetes."

The honest verdict​

"The same primitives already serve both" is true at the layer that matters most for day-to-day platform engineering — containers, declarative APIs, GitOps, policy, and observability really did converge, and a git-push PaaS gets that convergence for free. It is not yet true at the layer that matters most for actually hosting inference and agents at scale — GPU scheduling GA'd months ago, disaggregated serving needed its own new control plane, multi-agent orchestration still has no shared primitive, and CNCF's own certification program exists precisely because the gap is real. Cloud native is the substrate either way; the honest 2026 framing is that the substrate is finished at the foundation and still under construction at the layers a platform actually needs to serve a model.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. The same Cluster API foundation that provisions a CPU node pool today is the foundation a DRA-based GPU node pool gets added to tomorrow, not replaced. Star the repo on GitHub or deploy your first app today.