LiteLLM's Agent Platform Fixes Session Amnesia With One Postgres Table, Not a New Kubernetes Primitive

An AI agent is three tool calls into a multi-step deploy when the pod running its sandbox gets evicted — a node drains, a spot instance dies, an autoscaler decides to reschedule. The agent's next message arrives at a fresh pod with no memory of what it already did. Did the rollback finish? Was the health check called? The agent doesn't know, and neither does the platform that's supposed to be tracking it.

That's the specific failure mode BerriAI built LiteLLM Agent Platform to close. Shipped as an alpha on May 8, 2026, it's a self-hosted, Kubernetes-native layer on top of BerriAI's existing LiteLLM gateway — the same project with roughly 53,400 GitHub stars for its model-routing proxy, now extending into agent execution with a much younger companion repo sitting at about 1,100 stars. The pitch, in BerriAI's own words: "We built this because we wanted a managed agent solution, but fully self-hosted."

The interesting part isn't the pitch. It's the mechanism — and it's simpler than "Kubernetes-native session management" makes it sound.

The Trick: Split Identity From State​

LiteLLM Agent Platform doesn't invent a new way to checkpoint a running container. It doesn't snapshot process memory or freeze a filesystem mid-execution. Instead, it draws a hard line between two things that most agent-sandbox products conflate: the sandbox (an ephemeral, restartable compute unit) and the session (the durable record of what an agent has done and where it left off).

The sandbox side runs on kubernetes-sigs/agent-sandbox, a Kubernetes SIG Apps subproject built originally at Google and still at agents.x-k8s.io/v1alpha1 — alpha maturity. Its core Sandbox CRD gives a pod a stable identity and persistent storage independent of any one running instance, with companion CRDs (SandboxTemplate, SandboxClaim, SandboxWarmPool) handling reusable configs and pre-warmed pools for near-instant allocation. Google's own benchmarks claim up to 90% faster cold starts from warm pooling, with sub-second allocation off a warm pool. The isolation boundary itself is pluggable — gVisor or Kata Containers underneath, not bespoke sandboxing code.

The session side is just Postgres. A schema-migration init container runs on every startup to guarantee the database is in a known state before the app boots, and from there Postgres is the single source of truth for conversation history, tool-call state, and where an agent's task actually stands. When a pod restarts — planned upgrade, node failure, autoscaler churn — the sandbox is gone, but the session was never stored there in the first place. A new sandbox pod comes up, reads the same session row, and the agent picks up where it left off.

That's the whole trick: don't make the compute durable, make the state durable, and keep them in different systems. It's not a novel insight in distributed systems generally — it's the same reasoning behind storing web-app sessions in Redis instead of server memory — but it's a genuinely useful one applied specifically to the agent-sandbox category, where most products still treat "the sandbox" as the unit that has to survive.

It also explains why the agent-sandbox CRD's warm-pool mechanism matters here in a way it wouldn't for a stateless workload. If session state already lives outside the pod, the pod itself only has to be fast to replace, not long-lived. A SandboxWarmPool keeps a small number of pre-provisioned, isolated pods sitting idle so a fresh session (or a restarted one) can claim an already-running sandbox instead of paying full cold-start cost — Google's figures put that at up to 90% faster allocation, down to sub-second. Combine that with Postgres-backed session state and the practical effect is that a pod eviction becomes a latency blip an agent's next tool call absorbs, not an event the agent — or a human debugging it later — has to reason about at all.

Where That Puts It Against E2B, Daytona, and Modal​

The three most-cited hosted agent-sandbox vendors solve isolation differently, and none of them solve persistence the way LiteLLM Agent Platform does — because none of them are Kubernetes-native to begin with.

The pattern: E2B, Daytona, and Modal all compete on cold-start latency and isolation depth for a single sandbox instance. None of them ship a first-class concept of a session that outlives the sandbox it started in, and none of them have a per-team, multi-tenant isolation model — you get one sandbox per request, and if you want continuity across restarts, you build that yourself on top of their API.

That's not a knock on those three — they're optimized for a different job: spin up a sandbox, run some untrusted code, tear it down, repeat, as fast as possible. LiteLLM Agent Platform is optimized for the opposite shape: a long-lived agent identity (a team, a project) that keeps talking to the same logical session across however many sandbox pods actually run underneath it over the session's lifetime.

What's Still Genuinely Alpha​

BerriAI is upfront that this is pre-v0 and experimental — APIs are expected to change based on early feedback, and a few concrete gaps back that framing up:

• MCP transport is HTTP-only for now — no multi-server MCP aggregation behind a single /mcp endpoint yet, which matters if an agent needs to reach several tool servers through one connection.
• No documented GPU-in-sandbox support. Modal is the one hosted option in this comparison that explicitly supports GPU workloads inside a sandbox; nothing in LiteLLM Agent Platform's current docs addresses running a model inference step inside its own sandboxes.
• No published multi-node scaling guidance beyond the kind-for-dev, EKS-for-prod split — reasonable for an alpha, but a gap if you're planning to run this at a fleet size where a single management cluster's agent-sandbox controller becomes the bottleneck.

There's also a trust footnote worth pricing in before self-hosting any part of BerriAI's stack in production: the parent LiteLLM proxy project had a rough 2026 on the security front — a supply-chain compromise via hijacked maintainer PyPI credentials in March, and a CVSS 9.9 auth-bypass chain plus a SQL-injection CVE (exploited within 36 hours of disclosure) in April and May. Neither incident touched the Agent Platform code specifically, but they're the kind of history that argues for pinning versions and reading changelogs rather than auto-updating a self-hosted deploy-authority component.

The Actual Lesson for a Deploy-From-Chat Platform​

Here's the part that matters beyond BerriAI's own roadmap. Any platform whose MCP server holds real authority — deploy, rollback, read logs — over tenant infrastructure eventually has to answer the same question LiteLLM Agent Platform answered: what happens to an in-progress agent session when the pod underneath it dies mid-task?

The naive answer is "don't let the pod die," which doesn't survive contact with a real fleet — nodes drain, autoscalers reschedule, upgrades roll. The answer LiteLLM Agent Platform landed on is more durable: never make session continuity depend on one pod's uptime in the first place. Store the session's state — which deploy is in flight, what the last health check returned, whether a rollback was already triggered — in a database that outlives any individual sandbox, and treat the sandbox pod itself as fully disposable.

That's a smaller design change than it sounds like, and it's the one a deploy-from-chat MCP server needs before it can honestly claim an agent's bex deploy → health-check → rollback sequence survives the exact kind of infrastructure churn a self-hosted Cluster-API fleet produces routinely. The sandbox identity can live in whatever Kubernetes-native object makes sense for the platform; the session state doesn't belong there at all.

It also reframes what "audit trail" should mean for an agent with deploy authority. If session state is a Postgres row rather than something reconstructed from pod logs after the fact, "what did the agent already do" stops being a forensic question you answer by grepping kubectl history and becomes a query you run against a table that was the source of truth the whole time — the same row the agent itself was reading to decide its next action. That's a stronger guarantee than "we log everything and hope the logs survived the same restart the session state didn't," and it's the kind of guarantee worth building in before an MCP server's tool calls start carrying real production authority, not retrofitting after the first pod eviction corrupts an in-flight rollback.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with an MCP surface built for an agent to operate, not just deploy. Star the repo on GitHub or run your first agent-operated deploy today.

Sources​

• LiteLLM Managed Agents Platform — Alpha (BerriAI)
• GitHub — BerriAI/litellm-agent-platform
• GitHub — BerriAI/litellm
• GitHub — kubernetes-sigs/agent-sandbox
• Kubernetes Blog — Running Agents on Kubernetes With Agent Sandbox
• Google Open Source Blog — Why Kubernetes Needs a New Standard for Agent Execution
• MarkTechPost — Meet LiteLLM Agent Platform
• Daytona vs E2B vs Modal vs Vercel Sandbox 2026 comparison
• E2B Pricing
• The Hacker News — LiteLLM vulnerability chain
• The Hacker News — LiteLLM CVE-2026-42208 SQL injection