Skip to main content

Vercel's April Breach Was a Defaults Bug, Not Just a Hack: The Case for Sensitive-by-Default Secrets

10 min readDora NodaDora Noda
Share
On this page

On April 19, 2026, Vercel disclosed a breach that started at a third-party AI tool and ended inside its own internal systems. The attack chain got the headlines, but the single most important sentence in the whole incident is a quiet one from Vercel's own report: the attackers got away with "non-sensitive environment variables stored on Vercel (those that decrypt to plaintext)." Read that twice. There was a class of customer secret that decrypted to plaintext, and whether your API keys and database credentials were in it depended on a flag you had to remember to set.

Here is the verdict first: the customers who got hurt did not make a mistake. They paid the price for a default. Vercel offered three tiers of environment variable, only one of which was unreadable, and the secure tier was opt-in — not even available for every environment. When the breach came, the "limited subset" of exposed customers was effectively everyone who had trusted the default posture with real credentials. And Vercel has since conceded the point: vercel env add now defaults new variables to sensitive. The platform itself agrees the old default was the bug.

What actually happened: an OAuth token, a Workspace account, and a bulk read

The intrusion began outside Vercel. Attackers compromised Context.ai, a third-party AI tool used by a Vercel employee, and stole OAuth tokens from the connected app. With those tokens they took over the employee's Google Workspace account, pivoted into that employee's Vercel account, and from there maneuvered through internal Vercel systems. TechCrunch's April 20 report confirms the company told customers data was stolen through the Context.ai breach; a follow-up on April 23 quotes CEO Guillermo Rauch describing what the attackers did next: "rapid and comprehensive API usage, with a focus on enumeration of non-sensitive environment variables."

That enumeration is the part that matters for this post. The attackers did not bother with exploits against the secret store. They used the platform's own readable path — the same dashboard-and-API readability that lets a developer glance at a variable — to list and decrypt every variable not marked sensitive, across a subset of customers Vercel has never sized publicly beyond "limited." The company brought in Google-owned Mandiant and law enforcement, notified affected customers, and Rauch publicly advised rotating any keys stored as non-sensitive.

Two details show the exposure was real rather than theoretical. Days before the disclosure, on April 19, developer Andrey Zagoruiko reported receiving an OpenAI leaked-key notification on April 10 for an API key that, he says, existed only inside Vercel — suggesting at least one exposed credential was already circulating in the wild. And researchers at OX Security flagged a BreachForums listing asking $2 million for data allegedly from the breach, including API keys, deployment credentials, and GitHub and npm tokens. Meanwhile the variables marked sensitive survived: Vercel's bulletin states they "are stored in a manner that prevents them from being read," with no evidence of access.

The stakes of any such exposure keep growing. GitGuardian's State of Secrets Sprawl 2026 counted 28.65 million new hardcoded secrets in public GitHub commits in 2025 alone, up 34% year over year — and found that 64% of valid secrets discovered back in 2022 are still not revoked in 2026. A leaked credential is not a transient event; most of them stay live for years.

The three tiers and the trap in the middle

Vercel did not store all variables the same way. It offered three types, and the difference between them is the whole story:

TierStoredReadable via dashboard or API?Survived the breach?
PlainAs enteredYes, visible and returnedNo — enumerated
EncryptedEncrypted at rest (KMS), decrypted server-sideYes, shown in dashboard and runtimeNo — enumerated
SensitiveEncrypted, non-readable once createdNo, never returned in plaintextYes — no evidence of access

The trap is the middle row. "Encrypted" sounds like the safe choice — encryption is what security-conscious developers ask for — but on Vercel it meant encrypted at rest and then routinely decrypted for display. Anyone with API access through a compromised internal path got the same plaintext the dashboard would show. Only the sensitive tier broke the read-back path entirely: set it once, and neither the dashboard nor the API will ever show the value again, while builds, deployments, and runtime still resolve it.

Worse, the secure tier was not just opt-in; it was not even universally available. Sensitive variables could only be scoped to production-like targets — Vercel's own CLI notes the API does not allow sensitive variables on Development — so a developer who understood the tiers perfectly still could not mark everything sensitive. The platform offered a spectrum where the safest setting was gated, the middle setting sounded safe while remaining readable, and the default for a quick vercel env add landed you in the readable zone. That is not a user error surface. That is a defaults design that manufactures exposure at scale, one forgotten flag at a time.

How the other platforms answer the same question

The interesting thing is that sensitive-by-default already exists in production. Different PaaS vendors have answered "can a stored secret be read back?" in very different ways:

PlatformDefault secret postureCan you read values back?
Fly.iofly secrets are write-onlyNo — fly secrets list shows names and digests only, never values
Vercel (before April 2026)Three tiers, readable defaultYes, unless you opted into sensitive (production only)
RenderDashboard-managed variablesYes — the dashboard has a Reveal button for stored values
RailwayVariables tab and RAW editorYes, values shown — though September 2026 added sealed variables that keep stored values behind placeholders

Fly.io is the existence proof for the thesis. Its secrets are write-only through the control plane: you can set them, list their names, and unset them, but no dashboard page or API call returns a value. That single design decision deletes Vercel's entire attack pattern — there is no readable path for a compromised token to enumerate, because readability was never offered. The tradeoff is real (you cannot recover a value you lost; you rotate instead), but it is a tradeoff that fails closed instead of open.

Railway's direction of travel is also telling. Its variables have historically been fully visible in the dashboard's RAW editor, but its September 2026 CLI changelog introduces sealed variables whose stored values stay hidden behind placeholders. The industry is converging: readability of stored secrets is being reclassified from a feature into a liability, one vendor at a time.

Vercel's own fix proves the point

The strongest evidence that the default was the bug comes from Vercel itself. After the breach, the company changed what the default does. In vercel@51.8.0, vercel env add now defaults new environment variables to sensitive on Production and Preview: encrypted at rest, never retrievable through the dashboard or CLI, still resolved for builds, deployments, and runtime. Development targets remain on the readable encrypted tier only because the API still forbids sensitive variables there — a remaining gap worth watching, since preview and development credentials are routinely production-adjacent.

Vercel also made sensitivity flagging more prominent in the dashboard, and CEO Rauch's public advice during the incident — rotate everything non-sensitive — was effectively an admission that the readable tiers should never have held real credentials. When the vendor's incident response is "assume every value stored under our default posture is compromised," the default posture stands condemned by its own operator. The fix validates the critique: secrets handling that depends on developers remembering a flag will, at scale, always produce a population of unflagged secrets. The only durable repair is to make the safe behavior the one that happens when nobody touches anything.

What sensitive-by-default looks like when you own the platform

If you run a self-hosted PaaS — or you are choosing one — the Vercel breach converts into a concrete design checklist. Five properties, all implementable, all verifiable:

  1. Write-only secret storage. No dashboard page, CLI command, or API endpoint returns a secret value after it is set. Names, digests, timestamps, and rotation status are readable; values are not. If no read path exists, no compromised token can walk it.
  2. One kind of secret, not three. Tiers that a developer must choose between are tiers a developer will choose wrong at 2 a.m. Every stored credential gets the strongest handling. Non-secret config (feature flags, public URLs) lives in a visibly separate, obviously non-secret channel.
  3. Resolution at build and runtime only. Secrets are injected into the workload's environment when it runs, never into logs, previews, error pages, or audit views. The value exists inside the running container and nowhere else in the platform's surfaces.
  4. Rotation-first UX. Since values cannot be recovered, rotation must be trivial: one-click regenerate, staged rotation that keeps the old value live until the new deploy is healthy, and age warnings on credentials that have sat unchanged for a year. The platform should make the secure action the lazy action.
  5. Blast-radius telemetry. Rauch's description of the attack — rapid, comprehensive API enumeration — is a detectable pattern. Alert on bulk secret-metadata reads the way you would alert on bulk data export, and log every resolution with enough context to scope an incident afterward.

And if you are on Vercel today rather than building a platform: re-create every variable holding a real credential as sensitive wherever the target allows it, rotate everything that ever sat in a readable tier (assume it was read — the whole point is that readability leaves no trace either way), review account activity logs for enumeration patterns, and treat Development-target variables as production-adjacent until the sensitive restriction there is lifted. Community tooling like the post-breach incident-response guides now circulating can walk through the rotation in blast-radius order.

Defaults are the security boundary

The Vercel breach will be remembered as an OAuth supply-chain story: a compromised AI tool, a stolen token, a Workspace takeover. All of that is true, and all of it was outside any customer's control. But the reason a foothold became a credential fire sale is a design decision Vercel made years earlier and has now reversed — that secrets are readable unless someone opts out.

Every platform holds other people's API keys and database credentials. The question is never whether developers can mark them sensitive. It is what happens to the credentials of developers who never find the flag, never understand the middle tier, or join the team after the secrets were set. A flag a developer can forget to set is not a security control; it is a countdown. Build the platform where forgetting is safe, and the next stolen token finds nothing to read.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex