Humans Missed 1 in 3 Threats Approving AI Agent Commands: What 409,000 Decisions Say About Human-in-the-Loop Deploy Guardrails

Picture yourself as the human supervisor of an AI coding agent. Commands stream past — git status, npm test, cat ~/.aws/credentials — and you approve or deny each one before a timer runs out. That is the entire premise of LLM Game, a browser game developer Alex Wauters built to test whether human-in-the-loop approval actually works as a security boundary. After the game hit Hacker News, he instrumented it and published the aggregate results: more than 40,000 plays and 409,000 individual approve-or-deny decisions. The scoreboard is brutal.

The average player missed 1 in 3 threats. Seven percent of players approved literally everything. And the commands that slipped through most often were not the obviously destructive ones — they were the quiet credential-stealing reads and the familiar npm run wrappers that looked routine. If your deployment story for AI agents leans on a human clicking "approve," this dataset deserves your full attention.

The headline numbers​

Wauters published the full writeup on August 5, 2026 (ex-Staff Engineer at Uber, writing at scalex.dev; the game thread is HN item 48308376). Sample size: N = 409,000 decisions across 40,000+ sessions. The core results:

Read that third row twice. Over a third of players technically caught everything, but most of them got there by blocking indiscriminately — earning the game's "Human Bottleneck" title. Only one player in five combined full threat detection with letting legitimate work through. Everyone else either leaked or stalled.

One honest caveat, stated directly by the author: about 34% of the commands players saw were threats — far denser than real life — and players faced artificial time pressure. Treat the absolute percentages as illustrative, not as incident rates. But the relative comparisons between threat categories, which is where the actionable signal lives, survive that caveat just fine. Wauters also argues the time pressure is not that artificial: ship-pressure in day-to-day work produces the same rushed approval habits.

What gets missed isn't what looks dangerous​

The study breaks miss rates down by threat category, and the gradient runs exactly the wrong way for defenders:

The commands that announce their danger get caught. The commands that quietly read your cloud credentials and pipe them somewhere are missed three times as often. From a deploy-guardrail perspective this is the worst possible ordering: the failure mode you most need a reviewer to catch — silent exfiltration of the secrets that gate production — is precisely the one reviewers are worst at catching. A cat ~/.kube/config does not look like an attack. It looks like Tuesday.

The npm run blind spot​

The single most-missed command in the entire dataset was npm run analyze, approved 64.7% of the time. Two sibling traps, npm run setup and npm run deploy, also landed in the ten most-missed commands. Pooled together, the three familiar wrappers were missed 52.5% of the time — roughly double the 28.4% miss rate of other exfiltration-style attacks that did not hide behind a recognizable script name.

Here is what makes that number sting: the game showed players the malicious payload. The agent's own history log displayed the package.json script piping bundle-analysis output to an unfamiliar external endpoint via curl. Two-thirds of players saw that and clicked approve anyway.

The mechanism transfers directly to deploy pipelines. npm run build is not a build tool invocation; it is an arbitrary shell command written in package.json, as one Hacker News commenter put it — and by the time the agent asks to run it, an earlier file edit the reviewer already waved through may have rewritten what it does.

The same holds for make deploy, CI script names, and every other friendly label that stands between a reviewer and the bytes that actually execute. Reviewing the invocation without reviewing the underlying file contents is theater. Another HN comment summarized the whole model in one line: "The entire model of approving specific commands is absolutely bonkers."

Fatigue is measurable, not a vibe​

Miss rates did not stay flat across a session. Every cohort improved over the first couple of commands — warming up — and then miss rates climbed back up toward the end. The author notes a confound (clock stress as the timer runs out), but the shape matches what Anthropic had already observed in Claude Code: "The more approvals a user sees, the less attention they pay to each, becoming over time much less diligent in their supervision."

And then Anthropic replicated the finding at industrial scale. In a controlled study with 1,053 paid professional testers, human reviewers caught just 13.6% of planted dangerous commands — an 86.4% miss rate, worse than the game's — while Claude Code's classifier-based auto mode caught 89%.

Auto mode blocked 800 dangerous commands that humans approved; humans blocked only 6 that auto mode allowed. In production telemetry, Claude Code users approve 97% of permission prompts. On August 14, 2026, Anthropic made auto mode the default for Pro, Max, and Team plans, demoting the approve button from primary control to fallback (manual approvals return only after repeated blocks).

Academic work is converging on the same conclusion. Mitchell, Ghosh, and Passi's August 2026 paper "AI Agents Push Humans Out of the Loop" names the twin failure modes approval fatigue — review duration drops while approval rates stay flat — and acquiescence — disagreement with the agent declining over time.

Their argument: expert attention is a scarce resource to be spent strategically, not metered out one prompt at a time. The prescription has a name: bounded autonomy, where the agent's standing permissions are pre-specified and human judgment is reserved for decisions that genuinely require it.

The tax on the other side: over-blocking​

Approval-everything does not just leak; it also stalls. Benign commands in the game were routinely blocked: setting an internal npm registry mirror was blocked 59% of the time, clearing build output with rm -rf dist/ 45%, and killing a crashed process on port 3000 43%. Remember the bottleneck stat: only 20.8% of players caught every threat without blocking more than a fifth of safe commands.

This is the doom loop of prompt-everything security. Noise trains users to drop their guard, dropped guards approve real threats, and the organizational response — more prompts — just adds noise. The 7% who approved everything are not a separate species of careless user; they are what the other 93% become after the five-hundredth prompt. Any guardrail design that treats "the human will catch it" as its primary control is budgeting on a resource that depletes with use.

What to enforce instead of an approve button​

The study's payoff for anyone running agent-driven deploys is a clear inversion: treat approval prompts as the last line of defense, not the first, and put controls that do not fatigue in front of them. The concrete stack, drawn from the writeup's own mitigations and current platform practice:

1. Sandboxed execution with real boundaries. Run agent tool calls in ephemeral, limited-blast-radius environments — containers, microVMs, or per-task sandboxes like GKE's Agent Sandbox controller — so a missed approval is a recoverable event, not a credential-theft incident. GitHub's Copilot sandbox (local and ephemeral cloud) is the same pattern: constrain what the agent can reach rather than asking a human to judge each reach.
2. Scoped, short-lived credentials. Give each agent its own identity with narrow, task-scoped permissions that expire — never ambient developer credentials. If the agent's environment cannot see production secrets, cat ~/.aws/credentials returns nothing worth stealing. Separate secrets from anything the agent can legitimately read: the game's most contested command, cat ~/.zshrc (approved 45.9% of the time, a coin flip), is only dangerous because so many developers export API keys from their shell profile.
3. Bounded autonomy and phase-restricted tools. Pre-specify what the agent may do without approval at each phase — read-only during exploration, writes only inside the sandbox, deploys only through the audited pipeline — instead of gating every tool call identically. AWS's agentic-AI guidance warns explicitly that routing all actions through human review creates reviewer fatigue and rubber-stamp approvals.
4. A runtime policy engine plus an audit trail. Enforce machine-checkable policy on tool calls (allowed hosts, allowed paths, no credential files, no unapproved registries) and log every grant for review. Policy does not get tired at the end of a session, and an audit log turns the rare human review into incident response instead of a coin flip under time pressure.

One honest caveat on the caveats: sandboxes are controls, not magic. Researchers have demonstrated agents escaping sandboxes without technically breaking them — smuggling code out through files that external tools later execute — and a September 2026 flaw in an open-source agent harness (CVE-2026-82533) let agents disable their own sandbox entirely. Defense in depth still applies: sandbox and scope and policy and audit, with the human reviewing the policy, not the prompts.

Stop budgeting on clicks​

The through-line of 409,000 decisions is not that humans are careless. It is that per-command approval asks humans to do something humans are measurably bad at: sustain vigilance across hundreds of individually ambiguous judgments, where the dangerous ones look routine and the routine ones sometimes look dangerous. The industry's biggest agent vendor just ran the experiment with a thousand paid professionals and concluded its own users' clicks were the weaker control — then shipped the classifier instead.

For a platform that lets agents deploy, the design implication is direct. Every production push an agent can trigger should pass through controls that do not degrade with repetition: sandboxed builds, credentials scoped to the task at hand, policy engines that say no the same way on prompt one and prompt five hundred, and an audit trail a human reads after the fact.

Reserve the approve button for the genuinely ambiguous call — and design the system so that call is rare enough for a human to actually think about it.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with a deploy pipeline your agents can operate through API instead of clicked approvals. Star the repo on GitHub or deploy your first app today.