0.27s at 37%: What the August Sandbox Burst Benchmark Really Says About Cold-Start Claims

The fastest sandbox in the August burst benchmark goes from create() to a working shell in 0.27 seconds — when it works, which was 37 times out of 100. Every other provider in the same run completed all 100 launches, including the one with a 1.61-second median. If you buy sandboxes by the cold-start slide, that sentence should change how you read every number on it.

Here is the table, first, because the whole post is a guided reading of it. The dataset is ComputeSDK's open sandbox leaderboard, which measures Time to Interactive (TTI) — elapsed time from create() to the first successful command inside the sandbox — at 100 iterations per provider, launched concurrently in a single burst from a 4 vCPU host in Northern Virginia. These are the results from the August 21, 2026 run, as reported in MarkTechPost's August 27 write-up:

One sourcing note before the analysis, because attribution matters when numbers travel: the burst dataset above is ComputeSDK's, not Superagent's. Superagent's own benchmark work — a DX-and-pricing matrix built on vendor-published claims, plus an agent experience benchmark in which an AI coding agent live-tested ten providers — is a separate witness that shows up in section five. The two agree on the conclusion, which is exactly why both are worth your time.

Sequential is a different test than burst​

Daytona's 0.27-second median is real, and it is also the most misleading fast number in the table. On an earlier ComputeSDK provider-page run, Daytona created sandboxes at a 0.10-second median when launched one at a time. Under the August burst — the same harness, the same clock definition, just 100 concurrent creates instead of sequential ones — it posted the fastest median in the field and completed 37 of 100 attempts. Same vendor, same metric, different concurrency: a speedup headline becomes a capacity story.

As the MarkTechPost write-up puts it: a median you only reach on a third of your calls is not a latency number, it is a capacity number. Retry logic is not optional on any of these platforms — but there is a difference between retrying a tail-latency straggler and retrying two-thirds of your fleet's launches.

Now line that up against what the vendors actually publish. Daytona's pricing page advertises sub-90ms sandbox creation. E2B is commonly cited at roughly 150ms. Modal advertises sub-second cold starts for pre-cached containers. Superagent's matrix adds Blaxel at ~25ms standby resume.

None of these state concurrency, region, image size, or whether the clock stops at API acknowledgment or at the first executed command — and "pre-cached" and "standby resume" are doing enormous quiet work in two of those four claims.

Methodology doesn't just nudge these numbers; it dominates them. An independent sequential benchmark from March 2026 measured Daytona at roughly 1.2 seconds of startup plus 0.9 seconds of first-command execution — about 2.1 seconds time-to-first-command averaged over three runs, launched one at a time from the author's own location.

So for one vendor we now have four answers: sub-90ms on the pricing page, 0.10s sequential on the leaderboard, ~2.1s sequential from an independent harness, and 0.27s at 37% under burst. All four can be simultaneously honest. None of them is comparable to any other without the methodology footnote — which is precisely what the marketing slide omits.

Design against the tail, not the median​

If the Daytona row teaches "ask about the success rate," the Runloop row teaches "ask about the tail." Runloop's median (0.89s) and Modal's median (0.88s) are 10 milliseconds apart — a ranking gap of nothing. Runloop's P95 (3.27s) is 3.3x Modal's (1.00s). If your agent's UX budget is one second per tool call, the medians say these providers are twins and the P95s say one of them is still more than 3x over budget at one call in twenty.

Run the same budget math down the P99 column and the ranking inverts completely. At a strict one-second budget, no provider's P99 fits — not even Modal's 1.08s — which tells you the honest design conclusion: a per-tool-call sandbox create cannot sit inside a one-second interaction loop on any measured platform, and the median told you almost nothing about that. The number that sizes your timeouts, your retries, and your "hold the sandbox open across turns" decision is P99 under burst, full stop.

Cloudflare's 5.06-second median deserves a carve-out rather than a jeer: it is measuring a different product shape. Sandbox SDK sits on Cloudflare Containers, which schedules a container instance and boots an image — architecturally heavier than resuming a pre-warmed Firecracker VM. Cloudflare's own GA post is candid about the shape of the problem: booting a sandbox, cloning a repo, and running npm install takes about 30 seconds, while restoring the same environment from a backup takes about two. Judge a restore-optimized platform on restore, or hold sandboxes open per session instead of creating one per tool call — but don't rank its boot path against someone else's warm pool and call it analysis.

Why the architectures diverge under burst​

The burst table stops looking random once you sort it by isolation technology. E2B builds on Firecracker microVMs — true kernel-level isolation per sandbox, the same technology AWS uses under Lambda. Modal uses gVisor, Google's user-space kernel that intercepts syscalls: lighter than a microVM, a slightly weaker boundary, designed for exactly this serverless-untrusted-code use case. Daytona runs a container-based runtime it has open-sourced end to end (with VM and Windows classes alongside the container default), prioritizing creation speed and operator control over the hardening layer.

Map that onto the burst results and each row reads as an architecture expressing itself. E2B's 1.61s median with a tight 1.81s P99 is the Firecracker shape: every sandbox pays a real microVM-resume cost, but the cost is nearly constant and every launch completes. Modal's 0.88s median compressing to a 1.08s P99 is the gVisor-plus-warm-pool shape: cheap resume, tight tail, all 100 completions. Daytona's 0.27s-at-37% is the container-pool shape pushed past its warm capacity: the fast path is genuinely fast, and there wasn't enough of it for a 100-way burst.

None of these approaches is objectively wrong — the Tech Insider comparison makes the honest case that Firecracker is the safest default for fully untrusted third-party code, gVisor a reasonable middle ground when you control the agent but not every line it generates, and open container runtimes the right call when you need to audit or self-host the boundary itself. But "no wrong answer" is not "no observable behavior": identical medians can hide different failure modes, and the burst table plus the architecture note together tell you which failure mode you're buying.

A second witness: the agent that lived in all ten sandboxes​

Latency tables measure the sandbox from the outside. Superagent's agent experience benchmark measures it from the inside: an AI coding agent named Sol connected all ten providers to one TypeScript control plane with real credentials and tested stdout/stderr separation, non-zero exits, binary files, pause/resume, web servers, and repeated cleanup — the operations an unattended agent actually performs. Nine adapters were live-tested on August 27, 2026, with Freestyle following on September 10.

The ranking is worth quoting in full, because it inverts the burst table's speed order:

Daytona — the fastest median in the burst table — ranks fifth on usefulness to an actual agent, with the damning qualifier "highest maintenance": Sol reports repeated work across SDK changes, permissions, lifecycle inconsistencies, buffered output, and cleanup, plus a lost stderr stream on the first complete run. The verdict picks E2B for general coding, Modal for execution-heavy tasks, and Freestyle when pauseable full Linux VMs matter. The deciding factor, in Sol's words, is whether the agent can observe, modify, recover, and clean up the environment on its own.

Two independent benchmarks, two different methodologies, one shared moral: the number on the pricing page is the beginning of the evaluation, not the end of it. Burst behavior and lived agent experience both reorder the field — and they reorder it the same way.

A four-question decoder for any cold-start slide​

You don't need a leaderboard subscription to apply this. The next time a sandbox vendor — or a self-hosted-sandbox README — quotes you a cold-start number, ask four questions before comparing it to anything:

1. At what concurrency? Sequential creates measure the warm path; a 100-way burst measures warm path plus pool depth plus control-plane contention. Real agent fleets launch concurrently. If the number doesn't name its concurrency, assume sequential and discount accordingly.
2. Where does the clock stop? API acknowledgment ("sandbox creation") is not first successful command ("time to interactive"), and neither is a real task. The gap between Daytona's sub-90ms creation claim and the ~2.1s independent time-to-first-command is mostly this question, unanswered.
3. Is the region and image pinned? A cached micro image in the vendor's best region is the flattering case. Your agent's 2GB image with system packages in your region is the typical case. The leaderboard pins Northern Virginia and a fixed harness; your evaluation should pin yours.
4. Where are the success rate and the tail? Median without P95/P99 is a coin flip described by one side. Median without a success rate at burst concurrency may be a capacity number wearing a latency costume. No tail, no trust.

If you want to reproduce the leaderboard's discipline yourself, the recipe from the write-up is short: define the task your agent actually runs — install pandas, read a CSV, plot it, return a PNG — not echo hello. Time four checkpoints separately (create, first command, dependencies, task result). Run 100 iterations sequential, then 100 concurrent, and publish both series with median/P95/P99, because they answer different questions. Report TTI and task time separately: vendors optimize the first and your readers care about the second.

That recipe is also, not coincidentally, the acceptance test for the last section's claim. Anyone can dispute a ranking; what you can't dispute is a pinned harness with both series published. Run it against your shortlist before you sign — and keep running it, because pool depths and warm ratios change without a changelog entry.

The bar a self-hosted pool has to clear​

Everything above cuts both ways. If you run your own sandbox pool on owned hardware — Firecracker microVMs on Hetzner with KVM, say — you don't get to quote sequential restore latency and claim parity either. The bar is the concurrent one: 100-way burst, real image, real task, P99 and success rate published. Here is what the public reference points say that bar looks like:

• Snapshot restore, not boot, is the mechanism. NumaVM's March 2026 Firecracker measurements put bare VMM boot at ~38ms, full cold boot (host setup plus Firecracker plus Linux) at 1.1s, and snapshot restore at ~125ms. AWS's published posture is ~100ms microVM starts at thousands per second per host. If your pool boots kernels per request, you already lost to every row in the table; if it restores snapshots, you're in the game.
• Tens of milliseconds is the demonstrated class. Self-hostable Firecracker-forking projects report P50 restore latency around 27ms on a bare-metal reference node with a reproducible bench script. That is the number your pool has to reproduce under burst at 100% success before it beats the hosted field — matching it sequentially proves the mechanism, not the capacity.
• The warm pool is the product. Every fast row in the burst table is a warm pool being measured, and the 37% row is a warm pool being exhausted. Size yours for your fleet's actual burst factor, pre-warm the images your agents really pull, and alert on pool depletion — the metric Daytona's median hid is the first one you should dashboard.

Run the four-checkpoint harness from the previous section against your pool, publish both series, and you have an honest comparison against 0.67s/0.88s/1.61s medians at 100% success — plus a P99 that tells you whether your agent loop holds sandboxes open or creates per call. That is a stronger claim than any vendor slide, because it comes with the methodology footnote attached.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Agent sandboxes are just the sharpest edge of the same question every self-hosted platform answers: what does this cost, and what breaks first under burst? Star the repo on GitHub or deploy your first app today.