Skip to main content

Your Secrets Were Never in the Logs: How Mini Shai-Hulud Read Them Out of Runner Memory

11 min readDora NodaDora Noda
Share
On this page

On May 11, 2026, between 19:20 and 19:26 UTC, an attacker published 84 malicious versions across 42 @tanstack/* npm packages — six minutes, two versions per package, into one of the most depended-upon namespaces in frontend development (TanStack post-mortem). Here is the paradox that should freeze your blood: the release workflow runs that "produced" those publishes both show a red X. Tests failed, so the workflow's own Publish Packages step was skipped — and the packages were published anyway, each carrying a valid SLSA Build Level 3 provenance attestation (StepSecurity).

No npm token was stolen. The publish workflow was never modified. Detection came from outside: StepSecurity researcher Ashish Kurmi opened TanStack/router#7383 roughly 20 minutes after the first malicious publish, with a full technical writeup in hand. TanStack's own post-mortem is admirably blunt about the rest: three vulnerabilities chained together, each necessary, none sufficient alone.

This post walks that chain — and then stops at the third link, because it is the one that breaks the mental model most teams use to reason about CI secret safety. The worm did not read secrets from disk, from the environment, or from logs. It read them out of the GitHub Actions runner process's own memory, through /proc, grepping for the runner's internal secret representation. Every defense you can name — log masking, token scoping, "that secret is never even referenced in the YAML" — was in place. None of them look at process memory.

The initial access was a textbook Pwn Request, executed with patience. On May 10, the attacker created zblgg/configuration, a fork of TanStack/router deliberately renamed to evade fork-list searches, then authored a malicious commit on it under a fabricated claude <claude@users.noreply.github.com> identity — not the real Anthropic Claude, a forged no-reply address (timeline). The commit added packages/history/vite_setup.mjs, a roughly 30,000-line bundled payload, with a [skip ci] message prefix to suppress CI on push.

On May 11 at 10:49 UTC, pull request #7378 ("WIP: simplify history build") opened against TanStack/router's main branch. Two workflows using the pull_request_target trigger — bundle-size.yml and labeler.yml — auto-ran with no first-time-contributor approval, because pull_request_target bypasses that gate. (The pr.yml workflow, which uses the safer pull_request trigger, stayed blocked on the approval that never came.) The benchmark job checked out refs/pull/7378/merge — the attacker's merged code — and ran pnpm install plus a build. That executed vite_setup.mjs.

The payload's job was not to phone home yet. It was to poison the build cache. It wrote attacker-controlled data into the pnpm store directory under exactly the key the legitimate release.yml workflow would later compute and look up (Linux-pnpm-store-${hashFiles(...)}), and when the job ended, actions/cache's post-step saved the now-poisoned 1.1 GB store into TanStack/router's cache scope. The attacker then force-pushed the PR branch back to main's HEAD — making the visible PR a zero-file no-op — closed it, and deleted the branch. The poison persisted in the cache, invisible in the repo.

That evening, two release runs restored the poisoned entry, exactly as designed. Both failed their tests. And both published malware anyway — because by then, attacker-controlled binaries were on the release runner's disk, running with the release workflow's identity. TanStack's post-mortem names the two facts the workflow's author missed: cache writes use a runner-internal token that permissions: contents: read does not gate, and cache scope is shared per-repo between pull_request_target runs and pushes to main. This was not a TanStack-specific bug; it is the cache-poisoning class Adnan Khan documented in 2024, crossed with the Pwn Request pattern GitHub Security Lab has warned about for years.

Here is the step the TODO list for this post called "structurally novel," and it deserves the phrase. The release workflow legitimately declares id-token: write so it can publish to npm via OIDC trusted publishing. The GitHub Actions runner mints that OIDC token lazily, in memory, when the permission is set. The malware — a 2.3 MB single-line obfuscated router_init.js smuggled into each tarball — ran during the test phase, found the Runner.Worker process by scanning /proc/[pid]/cmdline, dumped its readable memory via /proc/[pid]/maps plus /proc/[pid]/mem, and piped the dump through this:

bash
sudo python3 | tr -d '\0' \
  | grep -aoE '"[^"]+":{"value":"[^"]*","isSecret":true}' \
  | sort -u

That regex matches the runner's in-memory JSON representation of a secret. Every repository, environment, and organization secret configured for the workflow sits in Runner.Worker's memory in precisely that shape — in plaintext, whether or not any step references it, whether or not it would ever be masked in a log. The scraper, decrypted by StepSecurity as "Payload 5" from the malware's AES-256-GCM layer, runs only on Linux GitHub Actions runners and skips the short-lived default github_token deliberately: the attacker wants long-lived credentials and the ambient OIDC token (Upwind's dissection).

The OIDC token is what resolves the opening paradox. The malware POSTed it straight to registry.npmjs.org's OIDC token-exchange endpoint, got back per-package publish tokens, and published the infected tarballs directly — bypassing the workflow's skipped Publish Packages step entirely. It then used the same stolen OIDC token against the Sigstore stack (Fulcio for a signing certificate, Rekor for the transparency entry) to mint SLSA v1 provenance with the standard GitHub Actions build type. The attestations are genuine: the pipeline really did build those artifacts. That is what makes this the first documented npm worm producing validly-attested malicious packages — and what makes StepSecurity's summary sentence the most important one written about this incident: "SLSA provenance confirms which pipeline produced the artifact, not whether the pipeline was behaving as intended."

One more uncomfortable fact before we move on: the memory-dump script was verbatim published tradecraft, attribution comment included. It is the same technique used in the tj-actions/changed-files compromise of March 2025 (CVE-2025-30066), where retagged action versions dumped runner memory — base64-encoded, specifically to defeat GitHub's literal-value log masking — into the workflow logs of more than 23,000 repositories. TeamPCP did not invent a capability. They industrialized a fourteen-month-old proof of concept into a self-propagating worm.

What you thought protected you, and what it actually protects​

The reason link three lands so hard is that every standard CI hygiene control answers a different question than "is the secret readable from the runner's address space?" Walk the table honestly:

Defense you hadWhat it actually guaranteesWhat the worm did instead
Log masking (***)The literal secret value won't print in logsNever looked at logs; read /proc/<pid>/mem
permissions: contents: readThe GITHUB_TOKEN can't writeDidn't need it; cache writes and OIDC minting use other tokens
"Secret never referenced in YAML"No step receives it as inputScraped all configured secrets from worker memory
Short OIDC token expiryA stolen token dies quicklyUsed it within seconds, inside the same run
Token scoping / least privilegeNarrows what each credential unlocksScoped tokens still live in the same memory
SLSA Build L3 provenanceAttests which pipeline built the artifactBuilt through that pipeline, then signed honestly

The pattern across the right-hand column is that every defense assumes the adversary is outside the runner looking in — at logs, at the network, at the registry. A payload executing on the runner during your release job is inside the trust boundary all of those controls assume. TanStack's post-mortem states the OIDC half of this flatly: "OIDC trusted-publisher binding has no per-publish review. Once configured, any code path in the workflow can mint a publish-capable token."

Why one bad install poisons the whole fleet​

The TanStack wave was the seed, not the campaign. The worm self-propagates: each infected host enumerates every package its victim maintains and republishes them with the same injection. Within 48 hours the count stood at roughly 172 packages across npm and PyPI in more than 400 malicious versions, with over 518 million cumulative downloads according to OX Security data (The Hacker News). The victim list grew to include Mistral AI, UiPath, the OpenSearch JavaScript client, Guardrails AI, Squawk, the AntV ecosystem wave on May 19, RedHat cloud-services packages, and — on May 18 — the Nx Console VS Code extension, pivoting the campaign from registries into the IDE channel. The long tail is still surfacing: CrowdSec disclosed that attackers used a stolen OAuth token to copy some 170 of its private GitHub repositories on May 22, an incident it did not discover until the archive appeared on a cybercrime forum in September (Dark Reading).

Build-pipeline hygiene is fleet hygiene. One npm install of a poisoned version on one machine harvested credentials from more than a hundred file paths — cloud metadata endpoints, Kubernetes service-account tokens, Vault tokens, ~/.npmrc, SSH keys — installed persistence hooks in Claude Code, VS Code, and OS services, and exfiltrated over the Session messenger file-upload network, which has no attacker-controlled C2 domain to block (StepSecurity payload analysis). Treat every install host as compromised is not paranoia here; it is TanStack's own stated guidance for anyone who installed an affected version on May 11.

What to change on Monday morning​

Map one fix to each link, in the order an attacker would re-walk them:

  1. Kill the Pwn Request surface. Audit every pull_request_target workflow: it must never check out and execute fork-controlled refs. Labeling and commenting are fine; checkout of the merge ref plus a build step is the vulnerability. While you are in there, pin every third-party action to a full commit SHA — floating refs are standing supply-chain risk independent of this incident.
  2. Isolate caches by trust boundary. Never let a key computed in a fork-PR context collide with a key a release workflow restores. Scope cache keys per ref or per trust tier, and treat actions/cache restore in a release job as code execution, because after May 11 it demonstrably is.
  3. Assume the runner's memory is readable. Restrict id-token: write to exactly the jobs that publish, scope npm trusted publishers as narrowly as the registry allows, and keep long-lived secrets out of release-job environments entirely — if a secret is configured for the workflow, a process on that runner can read it. Prefer credentials minted at the moment of use over credentials sitting in memory for the whole run.
  4. Run untrusted builds on ephemeral, egress-filtered runners. Single-job ephemeral runners (JIT or ARC scale sets) bound persistence; an egress allowlist turns "exfiltrate to a messenger CDN" into a blocked connection and an alert. StepSecurity's detection of this campaign came from network telemetry showing npm install opening connections to anything other than the registry.
  5. Monitor your own publishes. TanStack's sharpest self-criticism: "No internal alerting. We learned about the compromise from a third party." Alert on every publish from your scope — unexpected version, unexpected workflow run, unexpected hour — and rehearse the deprecation runbook, because npm's no-unpublish-with-dependents policy means you deprecate first and wait on npm security for tarball removal.

If you run the build farm, not just a repo​

Everything above is advice for tenants. If you operate the build system itself — a self-hosted PaaS, an internal platform team, a CI vendor — link three indicts the architecture, not the configuration. The GitHub Actions runner is a long-lived process that preloads every secret a job might need and holds them in one address space that any code running on the box can ptrace-equivalent read via /proc. "The secret was never logged or written to a file" was never a security boundary; May 2026 just made that undeniable at scale.

The fix is a different secret-injection model, and it has three parts. First, mint credentials at the point of use: a build step that needs registry access should receive a token created seconds ago for that step alone — short-lived, narrowly scoped, ideally bound to the step's identity — not inherit an environment where every credential for the whole pipeline is already warm in memory. Second, isolate build steps from each other as processes: run each step in its own mount, pid, and user namespace (or its own microVM) so a compromised test step cannot open the memory of the process that holds the publish credential. Third, make the publish path a separate trust domain: the job that can write to your registry should share no filesystem, no cache, and no secret set with the job that executes untrusted build scripts — TanStack's chain worked precisely because one runner, one cache, and one id-token: write spanned all three.

None of this is exotic. Ephemeral per-job runners, per-step credential vending through an OIDC broker with audience and subject restrictions, and artifact promotion gates between "built it" and "shipped it" are all established patterns. What Mini Shai-Hulud changed is their priority: they are no longer defense-in-depth for the paranoid. They are the minimum credible answer to an attack technique that is now published, productized, and worming.

Your build farm is infrastructure you should own, not a black box you rent trust from. Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with the build pipeline visible and yours to harden. Star the repo on GitHub or deploy your first app today.

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex