Skip to main content

The Tailscale Kubernetes Operator Exposes kube-apiserver Without Touching the Public Internet

11 min readDora NodaDora Noda
Share
On this page

Shadowserver's internet-wide scan found roughly 454,000 Kubernetes API servers observable online — and about 381,000 of them, 84%, answered with 200 OK. Most weren't misconfigured on purpose. They're the default outcome of how managed and self-hosted clusters alike ship: kube-apiserver listens on a public endpoint, TLS on port 6443, and the security model is "we firewalled it to our office IP and the CI runner." If you operate a self-hosted fleet — say, Cluster-API-managed clusters on Hetzner — that IP allowlist is probably the single load-bearing wall between the internet and the API that can schedule arbitrary pods on your machines.

The Tailscale Kubernetes operator offers a structurally different answer: front the kube-apiserver with an in-cluster proxy that is only reachable over your WireGuard-meshed tailnet, and have every request carry the requester's Tailscale identity into Kubernetes RBAC. No public control-plane endpoint at all. Here is the swap at a glance:

IP allowlist on a public endpointTailnet-authenticated API proxy
Exposure surfacekube-apiserver publicly reachable on :6443; allowlist filters whoNo public listener; proxy reachable only from the tailnet
Unit of authenticationSource IP (pre-auth), then client cert/tokenPerson or tagged device identity, mapped into RBAC
Audit granularity"A request came from 203.0.113.7""alice@example.com deleted deployment X from device Y"
Admin on/offboardingEdit firewall rules per IP changeAdd/remove the user from the tailnet; RBAC binding follows identity
CI accessPin or NAT runner egress IPs, chase churnRunner joins tailnet as a tagged device with scoped RBAC
Break-glass pathThe public endpoint itselfMust be planned separately — the proxy dies with the cluster

The rest of this post walks through both columns honestly: what the proxy actually does, what identity-based RBAC and audit look like in practice, and the real operational bill — including why every admin now needs to be on the tailnet, and what a team already running self-hosted Headscale should not assume they get for free.

The Status Quo: Firewalling kube-apiserver by IP

On a typical Cluster API fleet on Hetzner, each cluster's control plane sits behind a load balancer with a public IPv4, serving kube-apiserver on 6443. A Hetzner Cloud firewall (or the CAPH HetznerCluster spec's network rules) allowlists the office IP range, each admin's home IP, and whatever egress IPs your CI provider publishes. Authentication is a client certificate or a token in a kubeconfig.

This works until it doesn't, in three recurring ways:

  • IP churn. Admins roam — home ISPs rotate addresses, people work from hotels and conference Wi-Fi. Hosted CI runners draw from egress pools that change without notice. Every change is a firewall edit, and the path of least resistance trends toward wider CIDR blocks. The allowlist decays toward 0.0.0.0/0 one pragmatic exception at a time.
  • Audit opacity. The kube-apiserver audit log records a source IP and a certificate common name. Behind an office NAT, five engineers share one IP; a long-lived admin kubeconfig copied to a laptop looks identical to the original. When something is deleted at 2 a.m., "a request from the office IP using the admin cert" is not an answer.
  • Still a public TLS surface. An allowlist narrows who can complete a TCP handshake, but the listener is still on the public internet. Anyone inside an allowlisted network — a compromised office workstation, a rogue CI job — reaches the API server directly. And any pre-auth vulnerability in the TLS or HTTP stack is exposed to every allowlisted network in full.

That last point is the structural one. The allowlist is perimeter security around a public endpoint; it never changes the fact that the endpoint is public.

How the Operator's API Server Proxy Actually Works

The Tailscale Kubernetes operator runs in your cluster as a deployment that joins your tailnet like any other node. Among its features is an API server proxy: a component that sits between tailnet clients and the kube-apiserver, accepting connections only over the tailnet and forwarding requests inward. Enable it at install time:

bash
helm upgrade --install tailscale-operator tailscale/tailscale-operator \
  --namespace=tailscale --create-namespace \
  --set-string apiServerProxyConfig.mode="true" \
  --wait

With the proxy running, a fleet operator points kubectl at it with one command:

bash
tailscale configure kubeconfig <proxy-hostname>

That generates a kubeconfig whose server is the proxy's tailnet hostname. There is no public address to connect to, so there is nothing to allowlist — and once the cluster's own firewall closes 6443 to the world, port scanners find nothing at all.

The proxy runs in two modes:

  • Auth mode (the interesting one): the proxy authenticates the caller by their Tailscale identity and impersonates that identity to the kube-apiserver. A human shows up as alice@example.com. A tagged machine — a CI runner, a deploy agent — shows up with its tailnet FQDN as the username and its tags (like tag:ci) as Kubernetes groups.
  • Noauth mode: the proxy forwards requests without adding authentication, for teams that already run an external auth layer (OIDC, certs) and only want the network-level exposure fixed.

For production, the proxy no longer has to share the operator pod's lifecycle: a dedicated ProxyGroup with spec.type: kube-apiserver runs it as a replicated StatefulSet, with the replicas advertising a single stable endpoint via Tailscale Services (currently beta) for high availability across pod restarts.

RBAC by Identity, Not Source IP

Here is the part that changes day-to-day operations. In auth mode, reaching the proxy over the tailnet grants zero default Kubernetes permissions. Authorization is plain Kubernetes RBAC, keyed to the impersonated identity:

bash
# One human, read-only:
kubectl create clusterrolebinding alice-view \
  --user="alice@example.com" --clusterrole=view
 
# Every device tagged tag:engineering, read-only:
kubectl create clusterrolebinding engineering-view \
  --group="tag:engineering" --clusterrole=view
 
# Your on-call platform admin:
kubectl create clusterrolebinding oncall-admin \
  --user="bob@example.com" --clusterrole=cluster-admin

Compare that with the allowlist world, where "access" was binary: your IP is on the list and your kubeconfig has the admin cert, or not. Now the network layer answers can you reach it at all (tailnet membership + Tailscale ACLs), and RBAC answers what can you do — per person, per machine, per tag. Tailscale grants can additionally map tailnet user groups onto Kubernetes groups, so group:platform-team in your tailnet policy can become a Kubernetes group you bind roles to, without per-user bindings.

Offboarding collapses to one action: remove the user from the tailnet (which your identity provider can drive), and both the network path and the impersonated identity disappear. No firewall edit, no cert revocation scramble.

The Audit Trail You Didn't Have

Because every proxied request carries a person or device identity, the audit story changes in kind, not just degree:

  • Kubernetes API audit logging (beta, rolled out across 2026): the proxy records individual API requests as structured, identity-enriched events — who, from which device, did what to which resource — and can export them to cold storage for long-term retention and compliance.
  • Session recording: kubectl exec, attach, and debug sessions through the proxy can be recorded in full by pointing the operator at a tailnet recorder node.

The before/after on a single log line: an allowlisted public endpoint gives you GET /api/v1/secrets from 203.0.113.7, user=admin-cert. The proxy gives you alice@example.com, from device alice-mbp, ran kubectl exec into pod payments-7d4f... — with the terminal session replayable. For a small platform team that has ever done incident forensics against NAT'd source IPs and a shared admin kubeconfig, this is the feature that justifies the migration by itself.

What It Costs You: The Honest Ledger

None of this is free, and the costs are structural, not fine print.

Everyone must be on the tailnet first. Every admin, every CI runner, every automation that talks to the API server needs a Tailscale client and a tailnet identity before it can do anything. For humans that's an install and an SSO login; for CI it means your runners join as ephemeral tagged devices — supported, but now your deploy pipeline has a dependency on tailnet enrollment working. The first day this bites is the day a contractor needs ten minutes of read-only access and the answer starts with "install this client and accept the invite."

You've adopted a coordination-plane dependency. The tailnet's control plane — device registration, key distribution, ACL and grant evaluation — is Tailscale's SaaS. Your data path stays peer-to-peer WireGuard and keeps working through a coordination-plane outage, but enrolling a new device or changing a grant does not. You removed a public endpoint and added an external dependency; that's a trade, and you should make it with your eyes open.

Several pieces are beta. The API server proxy itself is documented as beta, as are Tailscale Services (the HA endpoint mechanism) and the structured audit logging. Track them before betting the only admin path on them.

The proxy dies with the cluster. The proxy runs inside the cluster it fronts. If the cluster is unhealthy enough — CNI down, node failures, a bad upgrade — your access path fails exactly when you need it most. You must keep an out-of-band break-glass path: a cert-based kubeconfig reachable via SSH to a control-plane node, or a firewall rule you can flip on in an emergency. This matters double for a Cluster API management cluster: it's the thing that repairs the workload clusters, so its access path should not depend on any single workload-path mechanism.

"But I Already Run Headscale"

If your fleet already uses self-hosted Headscale, be precise about what transfers. Headscale reimplements the coordination plane: node registration, key exchange, ACL policy distribution. It does not reimplement Tailscale's product feature set around this operator:

Headscale replacesStays tied to Tailscale's plane
Node registration & key distributionGrants syntax mapping tailnet groups → Kubernetes groups
Basic ACL policy for tailnet reachabilityThe operator's supported auth-mode identity workflows
DERP relay configurationSession-recording and audit-log export pipeline, admin console

The operator is built and tested against Tailscale's control plane; Headscale compatibility for its features is community-effort, best-effort, and typically behind. The honest framing: Headscale answers "I want the mesh without the SaaS." The API-server-proxy story in this post — grants-driven impersonation, recorded sessions, exported audit events — is an answer to a different question, and today it assumes Tailscale's plane. A Headscale shop can still get the network win (no public 6443, reachability only over the mesh) with plain subnet routing, but should not assume the identity and audit layers come along.

What This Looks Like on a CAPI-Managed Hetzner Fleet

Putting it together for a Cluster-API-on-Hetzner fleet:

  1. One proxy per workload cluster. Install the operator into each workload cluster with the API server proxy enabled; give each proxy a hostname that encodes the cluster (k8s-prod-fsn1, k8s-staging-hel1). Close 6443 on the Hetzner firewall to everything except the nodes themselves.
  2. Tags as your fleet taxonomy. Tag admin devices and CI runners (tag:platform, tag:ci-deploy), and bind Kubernetes roles to those tags identically across clusters. RBAC manifests become part of your cluster template, so a freshly CAPI-provisioned cluster comes up with correct access on day one.
  3. Keep the management cluster special. Give it the proxy too — but keep a documented, tested, cert-based break-glass path over SSH, because this is the cluster that heals the others.
  4. Agents are just tagged devices. If AI agents deploy and operate apps on your fleet, this model fits them unusually well: each agent joins as a tagged device, gets exactly the RBAC its job needs, and every API call and exec session it performs is attributed to that agent in the audit log — not to a shared bot token from an allowlisted IP.

The 84% statistic from the top of this post is not a story about careless teams; it's a story about defaults. A public kube-apiserver endpoint is what you get unless you engineer something else. The Tailscale operator's proxy is one of the first mainstream options where "something else" costs a Helm flag and a kubeconfig command instead of a bastion architecture — as long as you go in with the ledger balanced: identity-keyed access and a real audit trail on one side; tailnet onboarding for every principal, a SaaS coordination plane, and a mandatory break-glass plan on the other.


Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with AI agents as first-class, auditable operators. Star the repo on GitHub or deploy your first app today.

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex