Before Agents Create Their Own Compute: A Capacity Budget for Agent Sandboxes on Kubernetes

An accounting agent entered a runaway execution loop, made more than 15,000 high-cost API calls in under an hour, and generated roughly $50,000 in cloud charges before anyone stopped it. That case study comes from Google Mandiant's September 2026 enterprise AI report, and it is the compute story every platform team should read twice: the same loop pointed at a sandbox-creation API does not burn API credits, it eats your nodes. One agent stuck in a create-and-retry cycle can fill every machine in a self-hosted fleet while the dashboards still look green.

Upstream Kubernetes is about to make sandbox creation fast and normal. The kubernetes-sigs/agent-sandbox project brings sandbox lifecycle into native primitives — Sandbox, SandboxTemplate, SandboxClaim, and SandboxWarmPool CRDs, plus suspend and resume via spec.operatingMode. That is genuinely good news for operators. But the moment creation becomes a single agent-callable API call, capacity stops being a planning spreadsheet and becomes a control-plane decision. This post works out that decision as a concrete budget: warm-pool reserve, maximum concurrent sandboxes, per-tenant caps, TTLs, and the admission rules that stop one loop from consuming the fleet.

The budget model, worked end to end​

Take a reference fleet of three Hetzner AX42 machines (8 vCPU and 64 GB RAM each, roughly €46 per month per box) managed by Cluster API. Total raw capacity is 24 vCPU and 192 GB of RAM, for about €138 a month — less than one runaway afternoon in the Mandiant case study. That comparison is the whole argument for doing this math before shipping a create button.

Start by reserving system overhead: 1 CPU and 4 GB per node for the kubelet, OS, networking, and daemonsets. That leaves 21 vCPU and 180 Gi usable. Size each sandbox at a request of 500m CPU and 2 Gi RAM, with a limit of 2 CPU and 2 Gi RAM. The request/limit split is deliberate: CPU is compressible, so limits can overcommit it roughly 4x, while RAM is not compressible under isolation, so memory gets no overcommit at all.

The fleet is CPU-bound: 42 possible sandboxes against a RAM ceiling of 90. Hold 10 warm sandboxes in reserve (the warm-pool section below explains the sizing), and tenants can concurrently claim 32. Copy this table, substitute your node size and sandbox footprint, and you have your budget. Every section below is one row of it, defended.

The Cluster API scope: elasticity within a ceiling​

A fixed three-node fleet is where the budget starts, not where it ends. Run the sandbox node pool as a MachineDeployment with min 3 and max 6 nodes behind cluster-autoscaler. Below the ceiling, a pending SandboxClaim triggers scale-out exactly like a pending pod. At the ceiling, claims queue and admission rejects with a retryable message instead of silently degrading every tenant on the floor.

The relationship that matters: quotas are the spend ceiling, autoscaling is elasticity within it. The autoscaler decides how fast capacity appears; the per-tenant quotas below decide how much of it any one tenant — or any one runaway loop — can hold. Neither works alone. Autoscaling without quotas turns a retry loop into a machine-ordering loop. Quotas without autoscaling turn a legitimate burst into rejected claims on a fleet that could have grown.

Warm pools without warm-pool sprawl​

Cold boot is the enemy of agent UX, and snapshots already solved it. E2B's Firecracker-based platform restores pre-warmed microVM snapshots in about 150 milliseconds, against roughly 125 milliseconds for a bare Firecracker boot with under 5 MiB of overhead. GKE's Agent Sandbox integration reports pre-warmed pools starting sandboxes up to 90% faster than legacy paths. The upstream SandboxWarmPool CRD exists to make that pattern declarative: a pool of ready sandboxes that claims draw from instead of booting from zero.

But a warm pool is reserved capacity wearing a friendly name, so size it like capacity. The rule is pool size equals peak arrival rate times claim latency, clamped between a minimum that absorbs normal jitter and a maximum that bounds the reserve. For the reference fleet: if peak load claims 2 sandboxes per second and a claim takes 5 seconds to bind and hand over, the pool needs 10 warm sandboxes — the number already subtracted in the budget table. Set the minimum at 5 (never fully drain the pool chasing efficiency) and the maximum at 15 (the pool must never eat more than a third of the CPU-bound ceiling). Recompute the arrival rate weekly; agents get popular faster than forecasts do.

TTLs and lifecycle: every sandbox dies on schedule​

Upstream gives you two lifecycle levers: shutdown timers on the Sandbox and suspend/resume through spec.operatingMode (Running versus Suspended, landed in May 2026). Use both, and make every value a default the tenant can shorten but never remove. An agent that finishes its task and forgets to clean up is the normal case, not the failure mode — the failure mode is the platform trusting the agent to clean up.

Suspended sandboxes hold disk, not RAM, which is why suspended retention can be generous while idle TTL must be strict. The 8-hour max lifetime deserves emphasis: the widely reported coding-agent loop that burned $4,200 across 240 retries in three hours would have kept going without a wall clock somewhere. For compute, the TTL is that wall clock.

Per-tenant quotas: the spend ceiling in YAML​

Each tenant gets a namespace with a ResourceQuota for aggregate caps and a LimitRange for per-sandbox defaults. The quota caps total CPU, memory, and object counts; the limit range forces every sandbox to carry explicit requests and limits so the scheduler never treats a limitless pod as a zero-cost pod. A minimal starting point for the reference fleet, with four tenants sharing 32 claimable sandboxes:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-sandbox-quota
spec:
  hard:
    requests.cpu: "4"        # 8 sandboxes x 500m
    requests.memory: 16Gi    # 8 sandboxes x 2 Gi
    limits.cpu: "16"
    limits.memory: 16Gi
    count/sandboxes.agents.x-k8s.io: "8"

And the limit range that forces explicit requests onto every sandbox:

apiVersion: v1
kind: LimitRange
metadata:
  name: sandbox-defaults
spec:
  limits:
    - type: Container
      default:
        cpu: 500m
        memory: 2Gi
      defaultRequest:
        cpu: 500m
        memory: 2Gi
      max:
        cpu: "2"
        memory: 2Gi

Four tenants at 8 sandboxes each exactly fills the 32 claimable slots, which is the point: the quotas sum to the budget, so no combination of tenants can exceed it even if the autoscaler grows the fleet. Note the object-count line capping the Sandbox CRD itself — quota on CPU alone still allows a loop to create hundreds of tiny sandboxes that exhaust API-server and scheduler attention without tripping a single resource counter.

Admission policy: the last line before the bill​

Quotas cap totals, but admission policy rejects the specific creations that should never have existed. Two ValidatingAdmissionPolicy rules, enforced with CEL, close the runaway loop:

1. Every Sandbox must carry a TTL. Reject any sandbox whose shutdown timer (or max-lifetime annotation, depending on which upstream field your version serves) is missing or exceeds the 8-hour platform maximum. No TTL, no sandbox — the API error is the guardrail.
2. No tenant exceeds its sandbox count, whatever the autoscaler is doing. Recompute the tenant's live sandbox count at admission time and reject creations past the quota's object-count ceiling, even if cluster capacity is currently free. This is what stops one agent loop from consuming every Hetzner node: the 9th concurrent sandbox in an 8-slot tenant fails closed at the API server, before a single millicore is scheduled.

As of mid-2026, none of the dominant agent stacks — LangChain, LangGraph, CrewAI, the OpenAI Agents SDK, the Claude Agent SDK — ship a built-in primitive that closes the runaway-cost loop. Until they do, the platform's admission chain is the only loop-breaker that runs before money is spent rather than after.

Sensitivity: the same budget at three sandbox sizes​

The reference numbers above assume a medium 500m/2 Gi sandbox. The answer moves with the footprint, so here is the range on the same three-node fleet with the same 10-sandbox warm reserve:

Two things to notice. First, the fleet stays CPU-bound at every size, which means buying RAM-heavier nodes would not move the ceiling — the next euro goes to cores, or to a fourth node. Second, large sandboxes cut claimable capacity by two-thirds, which is why the limit range caps per-sandbox size: without it, one tenant requesting large sandboxes reprices the whole fleet for everyone else.

Ship checklist​

Agent-callable sandbox creation is coming to your control plane whether you plan for it or not — upstream is building the primitives now. Before the first agent gets a create button:

1. Work the budget table for your nodes and sandbox size; know your claimable number.
2. Set MachineDeployment min/max and ceiling behavior (queue + reject, never silent degradation).
3. Size the warm pool from measured arrival rate, with min/max clamps.
4. Default every TTL (idle, max lifetime, suspended retention) and let tenants only shorten them.
5. Quota every tenant namespace and enforce TTL + count rules at admission.

Do that, and the next runaway loop becomes an API rejection in a log line instead of a line item on a bill.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Agent sandboxes are on the same roadmap: declarative, quota-first compute for the agents that will operate your apps. Star the repo on GitHub or deploy your first app today.