Skip to main content

Signing Every Build Means Nothing If Nothing Checks: Wiring Cosign Into a Git-Push PaaS

10 min readDora NodaDora Noda
Share
On this page

Kubernetes signs every one of its releases with Sigstore. The project moved KEP-3031 — signing release artifacts with Cosign's keyless flow — to beta back in the 1.26 release, and today verifying a Kubernetes release means checking a signature rooted in a public transparency log, not trusting a tarball because it came from the right URL. Meanwhile, the CleanStart 2025 supply-chain report found that 20% of that year's supply-chain attacks involved poisoned or unverified container images. Both facts are true at once, and the gap between them is the subject of this post: signing is now easy, and signing without anything verifying at deploy time is theater.

Here is the whole argument up front, because the punchline should not wait for section four. A git-push PaaS that wants meaningful supply-chain integrity needs two halves, and most teams only ever build the first. Half one: sign every tenant image at build time with Cosign keyless signing — no keys to generate, store, or rotate. Half two: verify every signature at admission time, before the image lands on a node. On a Cluster-API-managed fleet that second half is one Kyverno policy:

yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-tenant-images
spec:
  validationFailureAction: Audit
  background: false
  rules:
    - name: verify-build-signature
      match:
        any:
          - resources:
              kinds: [Pod]
      verifyImages:
        - imageRefs:
            - "registry.example.com/tenants/*"
          attestors:
            - count: 1
              entries:
                - keyless:
                    subject: "https://github.com/your-org/paas-builder/.github/workflows/build.yml@refs/heads/main"
                    issuer: "https://token.actions.githubusercontent.com"
                    rekor:
                      url: https://rekor.sigstore.dev

Start it in Audit, watch what would have been rejected, then flip to Enforce. The rest of this post earns that snippet: how keyless signing works, what the build-side half looks like in a real pipeline, which admission-time verifier to pick, and an honest accounting of what the loop stops, what it does not, and what it costs.

How keyless signing actually works (no keys to lose)

The reason teams avoided image signing for a decade was key management: generate a private key, guard it in CI, rotate it on a schedule nobody keeps, distribute the public half to every verifier. Cosign's keyless mode deletes that entire job by binding the signature to an identity the builder already has. The five steps:

  1. CI proves who it is. The build job presents its OpenID Connect token — GitHub Actions, GitLab CI, and most managed CI systems mint these natively — to Fulcio, Sigstore's certificate authority.
  2. Fulcio issues a short-lived certificate. The cert binds the OIDC identity (say, a specific workflow file at a specific ref) to a freshly generated keypair, and expires in minutes. There is no long-lived private key to steal because nothing long-lived exists.
  3. Cosign signs the image digest. The signature covers the immutable digest (sha256:...), not the mutable tag. Moving a tag cannot move trust.
  4. The signature ships with the image. Cosign pushes it to the OCI registry as a companion artifact next to the image, so any registry that speaks OCI can carry signatures without special support.
  5. Rekor records it publicly. The signature, certificate, and identity go into Rekor, Sigstore's transparency log. Verification later means re-checking the signature and confirming the log entry exists — the same tamper-evident trick Certificate Transparency brought to TLS.

The verifier's question is therefore never "do I have the right public key file" but "was this digest signed by the identity I trust, and is the proof in Rekor." That shift is what makes per-tenant-image signing operationally feasible: the platform never handles key material at all.

Half 1: signing every tenant build in a git-push pipeline

In a git-push PaaS — buildpacks or Dockerfiles turning a push into an image — the signing step belongs immediately after the push, in the same job, using the pipeline's own OIDC identity. A GitHub Actions-shaped example:

yaml
permissions:
  contents: read
  packages: write
  id-token: write   # required: Fulcio mints the cert from this token
 
steps:
  - name: Build and push
    id: build
    uses: docker/build-push-action@v6
    with:
      push: true
      tags: registry.example.com/tenants/acme-web:${{ github.sha }}
 
  - name: Sign by digest
    run: |
      cosign sign --yes "registry.example.com/tenants/acme-web@${{ steps.build.outputs.digest }}"

Three details matter more than the tooling choice:

Sign the digest, not the tag. Tags are pointers; a signature over a tag attests to whatever the tag happened to point at. Cosign binds to the digest, and the admission policy in the next section re-checks by digest too. If your deploy path resolves tags to digests late — at kubectl apply time rather than build time — pin the digest in the manifest first or the verification chain has a gap an attacker can drive a retagged image through.

Pin the workflow identity, not "anything from our org." The Kyverno subject in the opening snippet names one workflow file at one ref. That is deliberate. A subject pattern as broad as "any workflow in our org" lets a compromised demo repo sign images your production policy will accept. One builder workflow, one subject, one trust root — then the blast radius of a breached side project is zero production deploys.

One signature per pushed digest. If the pipeline pushes both amd64 and arm64 images plus a manifest list, sign each digest you intend to deploy. Forgetting the manifest-list digest while enforcing verification is the classic way to break every multi-arch deploy the day you flip to Enforce — which is exactly why the rollout in the next section starts in Audit.

Nothing here requires exotic infrastructure: Cosign is a single binary, Fulcio and Rekor are Sigstore's public-good services, and the only secret involved is the CI system's own OIDC token, which already exists. Teams routinely report this half taking an afternoon. It is the other half that takes the quarter.

Half 2: who verifies at deploy time (the part most teams skip)

An unsigned-image policy has three credible implementations on Kubernetes in 2026, and they differ in where the verification logic lives:

ApproachHow it verifiesStrengthsWatch out for
Kyverno verifyImagesNative rule type; the admission controller fetches the OCI signature artifact and checks it against attestors in the policyOne tool for all policy, not just signing; background scans can flag existing violators; large policy libraryA verifyImages rule checks signatures or attestations, never both in one rule; needs registry reachability at admission time
Sigstore policy-controller (ClusterImagePolicy)Purpose-built controller matching images against authorities (keyless, key, or static)Deepest Sigstore integration (bundles, attestations, predicate types); warn mode for rolloutAnother controller to run; namespaces opt in via the policy.sigstore.dev/include label, so unlabeled namespaces silently skip verification
Gatekeeper + RatifyGatekeeper calls out to Ratify through the external_data provider at admission; Ratify does the Cosign/Notation lookup and returns pass/failFits teams already standardized on OPA/Rego; Ratify's verifier plugins cover Cosign and NotationTwo moving parts plus Rego to write; the external-data round trip adds admission latency to size for

For a PaaS team starting from zero, Kyverno is the pragmatic default: the signing policy lives alongside every other tenant guardrail (resource limits, forbidden registries, required labels) instead of in a second policy system. The policy from the introduction is nearly the whole thing; the production version adds three refinements.

First, roll out in Audit, then enforce. Kyverno's Audit mode logs what would have been denied without denying it. Run it for a week, grep the policy reports for would-be violations, and you will find the skeletons: the monitoring agent pulled by tag from Docker Hub, the backup sidecar nobody owns, the third-party ingress image that was never signed by anyone. Each needs either a signature of its own or an explicit exclusion — discovering them from logs beats discovering them from a 3 a.m. page.

Second, decide the unsigned-image story before you enforce. Your tenants' images are signed by your builder, but no policy survives contact with nginx:stable from a public registry. The standard pattern is scoping: enforce verification on your tenant registry paths (registry.example.com/tenants/*), and handle everything else with a separate rule — either a pinned allowlist of third-party digests or a requirement that platform images come from your own mirror, where you re-sign on ingestion. What you must not do is leave the default-deny off the tenant path to accommodate one unsigned sidecar; scope the exception, not the rule.

Third, plan for the verifier's dependencies. Admission-time verification needs the registry (to fetch the signature artifact) and Rekor (to check the transparency entry) reachable from the admission controller. If Rekor has an outage during your deploy window, Enforce with no fallback means no deploys — the textbook fail-closed dilemma. Kyverno lets you scope failurePolicy per rule; the sane posture is fail-closed on the tenant path (an attacker who can block Rekor can deploy anything otherwise) and an honest runbook entry that says deploys pause when Sigstore's public infrastructure is down. Check Rekor availability the way you check your registry's, because at admission time it is load-bearing.

What this stops, what it does not, and what it costs

Enthusiasm for signing curdles fast without a threat model, so here is the honest table:

ThreatDoes sign-plus-verify stop it?
Image swapped or tampered with after build (registry compromise, tag mutability attack)Yes — the digest won't match any valid signature
Rogue image deployed by a leaked kubeconfig (attacker pushes their own image and applies it)Yes — no signature from the trusted builder identity, admission denies it
Malicious dependency inside a legitimately built image (the 2024–2025 malware wave: 512,847 malicious open-source packages detected in 2024 alone, up 156% year over year per Sonatype)No — signing attests to who built it, not what's in it; you still need scanning, SBOMs, and SLSA-style build provenance
Vulnerable base image or library with a CVE (45,777 CVEs reported in 2025, roughly 130 a day)No — a signed vulnerable image deploys cleanly; pair verification with admission-time CVE gates
Compromised builder workflow signing a malicious imagePartially — Rekor's transparency log means the betrayal is visible and attributable, which deters and aids forensics, but the image still verifies

The 2025 numbers explain why the "no" rows matter as much as the "yes" rows: with over a third of supply-chain attacks entering through compromised dependencies and ENISA's inaugural Threat Landscape 2025 flagging a surge in supply-chain attacks across European infrastructure, a signature that says "our pipeline built this" is necessary but not sufficient. Signing answers provenance; scanning and provenance attestations answer content. Ship all three, in that order of operations.

The costs are real but bounded: a few seconds of admission latency per pod creation (size the controller replicas for deploy bursts), a hard runtime dependency on Rekor and your registry at admission time, and the one-time archaeology of finding every unsigned image already running. Against that, the payoff is the end of an entire class of incident: after Enforce, "something reached the nodes that our pipeline never built" stops being a hypothesis you investigate and becomes a thing your cluster refuses to do.

Rollout checklist, in order: sign by digest in the builder with a pinned workflow identity; deploy the policy in Audit; clear the would-be violations (allowlist or re-sign third-party images); confirm Rekor and registry reachability from the admission controller with alerting; flip to Enforce on the tenant path first, platform paths second; then add the content layer (SBOM attestations, CVE gates) on top of a provenance foundation that finally holds.

The checkbox is the enemy

Kubernetes signing its own releases with Sigstore set the expectation: provenance you can verify beats provenance you take on faith. But the project's signing only matters because verifiers exist — release tooling, package managers, and operators who actually check. A PaaS that signs every tenant image and verifies none of them has built half a bridge and painted "bridge" on the sign. The admission policy is ten lines of YAML and a week of Audit logs. There is no cheaper supply-chain win on a self-hosted fleet.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex