Skip to main content
bex.co
419
Blog

Kubernetes 1.35 Makes Supplemental Groups Strict: Closing the Shared-Volume Leak Hiding in Every Tenant Image's /etc/group

9 min readDora NodaDora Noda
Kubernetes
Security
Self-Hosting
PaaS
Infrastructure
Share
On this page

Two tenants share a node. Tenant B's PVC gets fsGroup: 3000, so anything in group 3000 can read and write its volume. Tenant A never asked for group 3000 — nothing in their Pod spec mentions it. And yet, on every Kubernetes release before 1.35's default behavior gets a real alternative, Tenant A's container can end up in that group anyway, because their container image happened to ship an /etc/group line that claims GID 3000 for its own purposes.

That's not a hypothetical. It's the exact mechanism Kubernetes v1.35 was built to close, and it's worth walking through concretely, because "the kubelet merges group information from the image" sounds abstract right up until you see the two numbers that collide.

The Leak: A Worked Example​

Say Tenant B's app requests a shared volume, and the CSI driver assigns it group ownership via the Pod's securityContext:

yaml
# Tenant B's pod
spec:
  securityContext:
    fsGroup: 3000
  containers:
    - name: app
      image: tenant-b/app:latest

Nothing unusual — fsGroup: 3000 means any process with GID 3000 in its supplemental groups can access files on that volume. Standard multi-tenant PVC provisioning.

Now look at Tenant A's image. It's an unrelated app, built from an unrelated base image, and somewhere in its Dockerfile a RUN groupadd shared-storage or a vendored base image left this in /etc/group:

text
shared-storage:x:3000:appuser

GID 3000. Same number, no coordination, no malice required — just a base image that happened to claim a low, easily-collided GID for its own internal purposes. Under Kubernetes' default behavior, when the kubelet starts Tenant A's container, it doesn't just attach the GIDs from the Pod spec (fsGroup, supplementalGroups, runAsGroup). It also reads /etc/group from the image, finds every group the container's primary user belongs to, and merges those in too.

Tenant A's process ends up running with GID 3000 as a supplemental group — a GID it never declared, attached because an image file said so. If that Pod ever gets scheduled onto Tenant B's node and mounts anything touching the same GID space, it can read and write Tenant B's data.

Nobody exploited anything. No RBAC was misconfigured. The platform did exactly what Kubernetes has always done by default — and that default is the problem.

This isn't only a two-human-tenants scenario, either. Any self-hosted PaaS running AI-agent sandboxes on shared nodes has the same shape of risk, just with a less predictable adversary: an agent-generated image, or a base image an agent pulled from a registry without a human reading its Dockerfile first, is exactly the kind of untrusted-image supply chain where nobody audited /etc/group before it landed on a node next to someone else's data.

What Merge Actually Does, Mechanically​

This behavior has a name now: Merge, and it's the value supplementalGroupsPolicy takes when a Pod doesn't set the field at all. It's not a bug — it's intentional backward-compatible behavior, and it's been Kubernetes' only behavior since the project existed. The kubelet computes a container process's supplemental groups by combining two sources:

  1. Pod-declared groups — spec.securityContext.fsGroup, supplementalGroups, and each container's runAsGroup.
  2. Image-declared groups — every entry in the image's /etc/group where the container's primary user (from runAsUser or the image's default USER) is listed as a member.

Merge unions both sets. That union is precisely how a GID nobody put in the Pod spec — sourced entirely from a file baked into the image — ends up as a live supplemental group on the running process, with real filesystem consequences on any volume that GID happens to own.

Strict Mode: The API and the Enforcement​

Kubernetes v1.35 graduates the fix — first proposed as KEP-3619, alpha in v1.31, beta in v1.33 — to General Availability: a supplementalGroupsPolicy field on .spec.securityContext that takes the value Strict.

yaml
spec:
  securityContext:
    fsGroup: 3000
    supplementalGroupsPolicy: Strict
  containers:
    - name: app
      image: tenant-a/app:latest

Under Strict, the kubelet attaches only the GIDs explicitly named in fsGroup, supplementalGroups, and runAsGroup. Whatever /etc/group says inside the image is ignored entirely for supplemental-group computation. Tenant A's shared-storage:x:3000:appuser line becomes inert — it no longer grants anything, because nothing in the image gets consulted.

Two enforcement details matter for a platform running this in production, not just a lab:

  • Runtime and node support are required, and the kubelet checks. Strict needs containerd v2.0+ or CRI-O v1.31+. The kubelet publishes support via Node.Status.Features.SupplementalGroupsPolicy, and it will reject a Strict Pod scheduled to a node that doesn't advertise support — it fails at admission, not silently at runtime with the old merged behavior. In a fleet mid-upgrade with mixed node versions, that's a hard scheduling constraint to plan around, not just a soft warning.
  • The result is now auditable. Beta in v1.33 added .status.containerStatuses[].user.linux, which reports the actual uid, gid, and supplementalGroups the kubelet attached to the first process in the container. You can read a running Pod's status and see the exact group set it started with — including confirming a Strict Pod really did drop the image's phantom groups — instead of inferring it from securityContext and hoping the merge logic did what you expected.

The Compatibility Trap: Why Merge Stays Default​

Here's the part that makes Strict a migration, not a flag you flip fleet-wide on a Friday: some images rely on /etc/group membership on purpose, and Strict breaks them the same way it breaks Tenant A's accidental collision.

The official postgres image is a real, common example, not a cherry-picked edge case — this pattern (a base image predefining a service-specific group for its own runtime user) is standard across a large share of official and Bitnami-style images, not an outlier. It ships a postgres group in /etc/group, and the postgres user is a member by default. Under Merge, a Pod running that image picks up the postgres GID automatically — no Pod-spec changes needed, and any volume or file pre-owned by that GID (say, a data directory provisioned by an init container using the same image) just works.

Flip that Pod to Strict without changes, and the postgres GID silently stops being attached. The container still starts, still runs as the right UID — but it may lose read/write access to files it previously could touch, because the group membership it was implicitly relying on is gone. No error, no crash on deploy — just permission denied the next time that access path gets exercised, which on a platform with staged rollouts can be hours after the change shipped.

That asymmetry — Strict closes a real leak, but also silently drops group access some legitimate images depend on — is exactly why Kubernetes kept Merge as the default even at GA, and why a self-hosted PaaS can't treat Strict as a one-line security freebie. It's a migration with an audit step:

  1. Audit tenant base images for /etc/group dependencies. Grep the images your tenants actually run (and any platform-internal images) for /etc/group entries the primary user belongs to, and check whether anything on a mounted volume is owned by that GID rather than an explicitly-declared one.
  2. Re-declare any group the image needs, explicitly. If an image legitimately needs the postgres GID, add it to that Pod's supplementalGroups list before flipping to Strict — the goal is an explicit Pod-spec declaration, not an implicit image-file one.
  3. Gate the rollout on runtime version, then canary. Confirm every node in the pool reports Strict support (containerd v2.0+/CRI-O v1.31+) so the kubelet won't reject scheduling, then roll Strict out to a subset of tenant namespaces first and watch for permission-denied errors before making it the platform default.

That third step matters more than it looks on a fleet built incrementally on owned hardware rather than a single hyperscaler image: a node pool assembled over a year or two of Hetzner or bare-metal purchases can easily be running three or four different containerd versions at once. Strict's node-level rejection means the same Pod template can succeed on newer nodes and fail to schedule on older ones in the same cluster — worth surfacing as a scheduling constraint (a node selector or taint keyed off the runtime version) rather than discovering it as unexplained Pending Pods during rollout.

Pairing With User Namespaces GA​

supplementalGroupsPolicy: Strict solves a narrower problem than User Namespaces, which reached GA in Kubernetes v1.36 and remaps a container's root UID to an unprivileged host UID (hostUsers: false). The two primitives are orthogonal, not overlapping: User Namespaces controls whose UID a container root actually is on the host; Strict controls which GIDs an image is allowed to silently claim for itself, independent of any UID remapping. A container running under a fully remapped user namespace can still inherit an unwanted GID from its own /etc/group under Merge — the two features close different doors.

For a Cluster API fleet's Pod template defaults, that means both belong on together for tenant workloads, not one in place of the other: hostUsers: false so a container-root compromise doesn't map to host-root, and supplementalGroupsPolicy: Strict so an image can't grant itself access to a co-located tenant's shared volume through nothing more than a line in a text file it shipped with.

Default It On, With the Audit Done First​

The concrete recommendation for a multi-tenant, shared-storage self-hosted PaaS: default new tenant Pod templates to supplementalGroupsPolicy: Strict, run the three-step audit above against your current tenant image set before the switch goes fleet-wide, and keep Merge only where a specific, already-audited Pod genuinely needs implicit image-group behavior. Merge staying the platform-wide Kubernetes default is a compatibility decision for the whole ecosystem — it isn't a recommendation for what your own multi-tenant defaults should be once you've done the one-time work of checking which images actually need it.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Its Cluster API-managed fleets can default tenant Pod templates to supplementalGroupsPolicy: Strict and hostUsers: false out of the box. Star the repo on GitHub or deploy your first app today.

Sources​

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex
Back to all posts