GKE Labs' OpenRL Runs Fine-Tuning as a Kubernetes Workload: What Owning the Loop Actually Saves Over a Managed API

Every managed fine-tuning API charges you twice for the same GPU hour: once to rent the hardware, and again as a per-token markup for the privilege of not managing it yourself. Google's GKE Labs open-sourced OpenRL in June 2026 specifically to let a team skip the second charge — by running the fine-tuning loop as an ordinary workload on a Kubernetes cluster it already owns, rather than a vendor's job queue.

The number that matters: where the markup actually sits​

Take Together AI's own published rates, because comparing a vendor against itself isolates exactly one variable — the markup — instead of mixing in a different vendor's cheaper hardware. Together charges $0.48 per million tokens for LoRA supervised fine-tuning on a 16B-class base model through its managed API. That same company also rents you the raw hardware directly: a dedicated H100 endpoint at $6.49/hour.

Do the algebra on those two numbers, and you get the throughput a self-managed training job needs to clear before renting the GPU outright beats paying Together's own per-token price: $6.49 ÷ $0.48 per million tokens works out to roughly 13.5 million tokens per hour — about 3,750 tokens/second. Reported LoRA fine-tuning throughput for 7B-14B models on modern hardware commonly lands in the 5,000-20,000+ tokens/second range, depending on batch size, sequence length, and framework — which means the realistic case clears that bar comfortably, not marginally:

That's the honest shape of the claim "no per-token fine-tuning markup": it isn't that self-hosting is free, it's that you're paying for the GPU-hour once instead of paying for the GPU-hour and a multiplier on top of every token that crosses it. OpenRL is Google's answer to the obvious follow-up question — if the markup is avoidable, what replaces the vendor's job queue and training harness once you strip it out?

What OpenRL actually replaces​

OpenRL implements a Tinker-compatible API — a design pattern Google credits to Thinking Machines' Tinker framework — built around four primitives: move training data in and out, update model weights, generate samples for a reward model to score, and save weights when a run is done. That's the entire surface. Google's own framing of why this split works: "AI Researchers get full flexibility on their RL loop and infrastructure engineers can focus on scaling, orchestration, and reliability."

The practical result is a decoupling that matters more than it sounds: a researcher's reinforcement-learning loop — the Python code deciding what to train on, how to score outputs, when to stop — can run on a laptop, calling those four APIs against training infrastructure sitting on a Kubernetes cluster somewhere else entirely. Google's own post is explicit about this: "you do not have to run the RL loop directly on the machines with GPUs, you can simply run your RL loop on your Mac pointing to the training APIs running on a Kubernetes cluster/VMs." The GPUs, the checkpointing, the weight storage — all of it lives behind an API a laptop can call, the same shape as a managed fine-tuning endpoint, except the endpoint now points at infrastructure you provisioned instead of a vendor's.

That's also the entire reason OpenRL is a plausible drop-in for a Cluster-API-managed fleet rather than a GKE-exclusive feature: the project explicitly runs on "any standard Kubernetes cluster," and its Apache-2.0 license and 4-API surface don't reference anything GKE-specific in how a training job actually executes.

Concretely, a single RL iteration walks all four primitives in sequence: the researcher's loop calls the sampling API against the current weights to generate a batch of candidate outputs, scores those outputs against a reward function, feeds the scored batch back through the data-transfer API, calls the weight-update API to apply a training step against that batch, and — once the loop decides it's done — calls the save-weights API to persist the result. None of that sequence cares whether the cluster answering those four calls is GKE, a bare-metal Cluster API Provider Hetzner fleet, or a laptop's local Docker Desktop cluster during early development; the API contract is the same either way, which is precisely what makes it portable instead of another vendor-specific training harness to learn.

What "owning the loop" doesn't include yet​

Here's the reconciliation the number above needs before it means anything: OpenRL's own maintainers are explicit that today's release is scoped narrowly. "We have started with a simple architecture focussing on LoRA fine-tuning" — full parameter fine-tuning and multitenancy (running RL against several different base models on the same infrastructure at once) are both listed under "future steps," not shipped. And Google says plainly: "OpenRL is self-hosted and not a managed service." Everything a managed API's job queue used to handle invisibly — which node a job lands on, what happens when a run needs to be retried, whether two jobs can share a GPU without stepping on each other, dashboards for whether a training run is actually progressing — is now the cluster operator's problem, not a line item on someone else's bill.

That work isn't a per-job cost, though — it's a setup cost, paid mostly once: standing up GPU scheduling and job orchestration on your Cluster API fleet, wiring monitoring into whatever's already watching the rest of the platform, building the retry/checkpoint discipline into the training harness. That distinction is what decides whether the markup avoided in the table above is actually worth collecting. Fine-tune a model exactly once, and the engineering time to stand up scheduling and orchestration will almost certainly cost more than the markup you'd have paid a managed API for that single run — self-hosting a one-off job is a bad trade. But OpenRL's actual target use case is the opposite shape: reinforcement fine-tuning is iterative by design — train, score, retrain, repeat — meaning a team running this loop seriously isn't paying for one job, it's paying for dozens or hundreds of them against the same infrastructure. The setup cost gets paid once; the 1.3x-2.7x markup avoided in the table above compounds on every run after that. The breakeven question isn't "is this job worth self-hosting" — it's "how many training runs will we actually do," and for anything past a handful, the answer tips toward owning the loop.

Where the training job should actually run​

This is also where the temptation to reach for the wrong default shows up. A Cluster-API fleet built around owned Hetzner hardware already has a GPU option on the shelf — a flat-rate box like the GEX44 — and it would be a mistake to point OpenRL's training jobs at it by default. That box earns its keep on always-on, predictable inference traffic clearing roughly a third of the day in utilization; a fine-tuning run is the opposite shape, a short, spiky burst of GPU-hours that finishes and goes quiet. Provisioning a fixed monthly GPU box to sit mostly idle waiting for the next training run wastes exactly the flat-rate advantage that makes owned hardware worth it for steady traffic in the first place.

OpenRL's Kubernetes-native design is what makes the better answer possible without inventing new infrastructure: because it runs as an ordinary workload against the standard Kubernetes API, a Cluster-API-managed fleet can provision a short-lived, rented GPU node pool — on-demand or spot capacity from a hyperscaler, scaled to zero between runs — specifically for training jobs, structurally separate from the fleet's always-on, owned-hardware node pool serving tenant apps. The training workload rents exactly the GPU-hours it uses and nothing more; the app-hosting workload keeps the flat-rate box it actually benefits from. Same Cluster API control plane, two different node pools sized for two different traffic shapes — not one GPU strategy stretched to cover both.

What it means for a self-hosted platform's own boundaries​

For a platform whose pitch is a Render-compatible API a tenant (or an AI agent operating on their behalf) calls to deploy and run an app, OpenRL is a useful data point about where that pitch's boundary should sit. A fine-tuning job is, mechanically, just another workload that needs a container, a GPU, and an API to talk to — the same shape as any other tenant service a Cluster-API fleet already knows how to run. That's a reason to make sure a GPU-capable node pool is a provisioning option a tenant can reach, not a reason to start managing the fine-tuning job itself. OpenRL's own framing draws that same line for infrastructure engineers generally: it hands the researcher the training loop and hands the operator the scheduling — it doesn't collapse the two into one managed product. A self-hosted PaaS taking on a tenant's fine-tuning pipeline as a first-party managed feature would be repeating the exact per-token-markup pattern this whole comparison exists to avoid, just one layer up the stack.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, on a Cluster-API-managed fleet that can provision GPU node pools alongside ordinary app-hosting compute. Running your own OpenRL-based fine-tuning loop on that fleet is a workload the platform can host — not a managed feature it needs to build. Star the repo on GitHub or deploy your first app today.