Skip to main content

SPIRE Needs Your Fleet's Shape in Advance. An Orchestrator Spawning Sub-Agents Doesn't Have One.

8 min readDora NodaDora Noda
Share
On this page

An orchestrating agent on a deploy-from-chat platform gets a request to parallelize a build: lint, test, and provision three services at once. It spawns three sub-agent containers to do it. Each one needs to call back to the platform's own APIs, read a scoped secret, and report status — which means each one needs a cryptographic identity, not a shared static token baked into the image. SPIFFE/SPIRE is the 2026-standard answer for exactly this: short-lived, rotatable, non-human workload identity instead of a .env file full of long-lived keys.

Except SPIRE's core mechanism assumes something that isn't true here: that every workload it will ever issue an identity to has already been described to the server, in advance, by name or by pattern. That assumption holds fine for a known set of long-running services. It breaks the moment "how many sub-agents, of what shape, doing what" is a decision the orchestrator makes at request time — because SPIRE doesn't have to guess which sub-agent to trust, it has to be told, and nobody told it about the third one yet. The fix isn't a bigger registration pipeline. It's registering the pattern once instead of the instance every time — and that distinction is the whole post.


The model SPIRE was actually built for

SPIRE splits identity into two attestation steps. Node attestation happens once, when a SPIRE Agent starts on a machine — it proves to the SPIRE Server which node it's running on (cloud instance identity document, join token, whatever the platform plugin supports), and the server hands that node a set of registration entries it's authorized to serve. Workload attestation happens per-process: when something on that node asks the local Workload API for an identity, the SPIRE Agent inspects it — PID, container image hash, Kubernetes namespace/service account/pod labels via the kubelet API, whatever selectors the platform supports — and checks that evidence against registration entries scoped to that node.

The registration entry is the load-bearing object, and it's created on the server before any of this happens:

text
spire-server entry create \
  -parentID spiffe://bex.internal/spire/agent/node \
  -spiffeID  spiffe://bex.internal/orchestrator \
  -selector  k8s:ns:agents \
  -selector  k8s:sa:orchestrator

That single entry says: "any workload running in namespace agents under service account orchestrator gets the identity spiffe://bex.internal/orchestrator." It's a policy rule, written once, for a workload whose shape — namespace, service account, image — is known ahead of time because a human or a deploy pipeline created it deliberately. SPIRE doesn't attest first and register second; the entry has to exist, or attestation has nothing to match against and no SVID gets issued. That ordering is fine for the orchestrator itself. It's the problem for what the orchestrator spawns.

The registration race

Walk through what happens when the orchestrator above spawns sub-agent three. The pod comes up in namespace agents, under a service account scoped for sub-agent workloads, with a label carrying its instance ID — something like bex.co/agent-id: sub-agent-7f3ac2, generated at spawn time, never seen before, and gone in ninety seconds when the task finishes.

The SPIRE Agent on that node picks up the new workload, gathers its attestation data, and asks the server: does any registration entry's selectors match this? If the only entries on the server are the ones created ahead of time for known, named services — the orchestrator, the API gateway, the build cache — the answer is no. No matching entry, no SVID. The sub-agent has no identity to present when it calls back to the platform, reads its scoped secret, or reports status to the orchestrator. It's either stuck making unauthenticated calls (a hole you don't want) or it fails outright.

Two fixes teams reach for first both run into the same wall:

  • Register each sub-agent by exact ID before spawning it. The orchestrator calls the SPIRE Server admin API — entry create -spiffeID spiffe://bex.internal/agents/sub-agent-7f3ac2 ... — synchronously, then spawns the pod. It works, but it puts a privileged SPIRE Server write on the hot path of every sub-agent spawn, and it leaves a registration entry to clean up (or leak) for every ephemeral workload that ever ran. A fleet spawning dozens of sub-agents per user request accumulates thousands of entries a day, most of them describing something that no longer exists ten seconds after it was created.
  • Pre-register a fixed set of known "shapes." Register sub-agent-linter, sub-agent-test-runner, sub-agent-deployer ahead of time and reuse those identities. This survives until the orchestrator does something it hasn't done before — a new tool, a differently-parameterized fan-out, a role nobody anticipated at registration time — and then it's back to the same gap, because the fleet's shape isn't fixed; it's decided per request.

Both approaches are trying to make the instance known in advance. The instance is never going to be known in advance — that's the entire point of an orchestrator that decides its own fan-out at runtime. What can be known in advance is the pattern the instance will match.

The fix: register the pattern, not the instance

spire-controller-manager — the Kubernetes controller that reconciles SPIRE registration state from CRDs — ships a ClusterSPIFFEID resource for exactly this. Instead of a registration entry per workload, you declare a selector and a template once:

yaml
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterSPIFFEID
metadata:
  name: bex-sub-agents
spec:
  className: bex-spire
  spiffeIDTemplate: >-
    spiffe://{{ .TrustDomain }}/ns/{{ .PodMeta.Namespace }}/agent/{{ index .PodMeta.Labels "bex.co/agent-id" }}
  podSelector:
    matchLabels:
      bex.co/agent-role: sub-agent
  namespaceSelector:
    matchLabels:
      bex.co/tenant-workloads: "true"

This is one Kubernetes object, applied once. From here, the controller watches pod-create events across the cluster; any pod landing in a matching namespace with the label bex.co/agent-role: sub-agent gets reconciled into a registration entry automatically, with the SPIFFE ID rendered from that specific pod's own metadata via the template — PodMeta.Labels, PodMeta.Namespace, and friends are populated from the live pod object, not from anything the orchestrator had to tell SPIRE about ahead of time. The orchestrator's only job at spawn time is to set the label — cheap, local, no SPIRE API call — not to negotiate an identity for a workload that doesn't exist yet.

That's the actual resolution of "the fleet doesn't know its own shape in advance": SPIRE never needed to know the shape either. It needed a rule general enough to cover every shape the platform is willing to trust, evaluated against each pod as it appears. The orchestrating agent can spawn one sub-agent or fifty, with instance IDs it invents on the spot, and every one of them attests successfully the moment the SPIRE Agent on its node sees the pod — because the registration entry it needs was never instance-specific to begin with.

What this doesn't fix

Three things this pattern doesn't solve, and a platform team building on it needs to plan for the gap, not discover it in an incident:

It answers "who," not "should." A sub-agent that attests successfully and holds a valid SVID for spiffe://bex.internal/ns/agents/agent/sub-agent-7f3ac2 has a verified identity — that's all SPIFFE promises. Whether that specific identity should be allowed to call rollback on a specific tenant's app, or read a specific secret, is a separate authorization decision SPIRE has no opinion on. Label-based attestation actually makes this sharper, not softer: broadening the selector that grants an identity broadens who gets one, which is exactly the kind of policy change that needs its own review path, not a rubber stamp because "it's just an attestation rule."

The hourly-rotation number doesn't mean what it sounds like for short-lived workloads. SPIRE's default X.509-SVID TTL is one hour (default_x509_svid_ttl, overridable per entry), and production reference architectures citing "SVIDs rotate every hour" are describing exactly that default — a sane setting for a service that runs for days. A sub-agent that lives ninety seconds never actually experiences a rotation; it's issued one SVID at spawn and exits before the TTL is ever tested. The number to actually tune per entry isn't the rotation cadence, it's the TTL itself — short enough that a compromised or hung sub-agent's credential doesn't outlive its intended job by an order of magnitude, which for a workload with a two-minute expected lifetime might mean an -x509SVIDTTL set in minutes, not the hour-long server default built for something else.

Bare-metal and VM sub-agents don't get this for free. ClusterSPIFFEID and the k8s workload attestor are Kubernetes-native — they work because kubelet exposes pod metadata the SPIRE Agent can query. A platform that runs some workloads as Cluster API-managed Kubernetes pods and others as machines or containers outside that control plane (a bare-metal build runner on owned hardware, say) needs a different workload attestor for the non-k8s case — Unix process attestation, Docker attestation, or a custom plugin keyed to whatever the platform's own orchestration layer already knows about that workload. The pattern-over-instance principle carries over; the specific selectors and CRD don't.


None of this is a reason to avoid SPIFFE/SPIRE for agent identity — the alternative most platforms ship today is a long-lived API key baked into every sub-agent's environment, which is a worse problem than a registration-entry ceiling. It's a reason to build the registration layer for the fleet you'll actually have — one that decides its own shape per request — instead of the fleet SPIRE's tooling defaults to assuming: small, known, and mostly static.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with agents as first-class operators of the platform, not just callers of it. Identity for a fleet that doesn't know its own shape in advance is exactly the kind of infrastructure problem that belongs to the platform, not to whatever ad hoc credential-passing scheme an agent framework bolts on later. Star the repo on GitHub or deploy your first app today.

Related articles

Give your agents a chain backend

Autonomous agents hit RPC endpoints very differently than people do. See what bex router handles on their behalf.

Read the agents guide