Skip to main content

Skupper vs Submariner vs Istio Ambient: Picking the Cross-Cluster Link for a Hetzner-Split Fleet

11 min readDora NodaDora Noda
Share
On this page

Your first region was easy. One Cluster API fleet on Hetzner, one cluster, every tenant app a short hop away. Then a customer asked for data residency in a second region, or you decided Falkenstein alone was a single point of failure, and suddenly you own two clusters — say Falkenstein and Helsinki — with a tenant frontend in one calling a tenant API in the other across the public internet. That call needs encryption, identity, and a route. The question is which layer you buy it at.

Three answers dominate the conversation in 2026. Submariner joins the two networks at Layer 3 with encrypted tunnels. Skupper v2 interconnects the two clusters at Layer 7, exposing services without joining networks at all. And Istio's ambient multicluster, which went beta at KubeCon EU 2026, stretches one mesh across both networks. They solve the same problem with different failure modes, and for a git-push PaaS the choice matters most exactly where tenant traffic — not just your operator traffic — crosses the region boundary.

Here is the whole post in one table. The sections below it are the receipts.

Submariner (L3)Skupper v2 (L7)Istio ambient multicluster (beta)
Network layerLayer 3: encrypted tunnels between gateway nodesLayer 7: app-layer routers, no VPNMesh: mTLS via east-west gateways
Overlapping Pod CIDRsNeeds Globalnet remapping when both clusters share CIDRsNo coordination needed — networks stay separateMesh sits above pod networking; no remap
Gateway / HA modelDedicated gateway nodes per cluster; HA = active/passive gateway failoverRouters per site; links are ordinary outbound connectionsEast-west gateways per cluster + per-cluster control plane
mTLS identity across clustersTunnel encryption (IPsec/WireGuard), not workload identityPer-link TLS with token-claim bootstrapFull SPIFFE workload identity, mesh-wide policy
Control-plane couplingBroker cluster coordinates join; data plane is per-clusterNone — no shared control plane at allMulti-primary: every cluster runs istiod; waypoints synced by you
MaturityStable, years in productionv2 stable line (v1 maintained, no new features)Multi-network multicluster beta, March 2026

If you only take one row away: Submariner merges networks, Skupper links services, Istio merges meshes. Pick the merge you actually want, because each one drags its own operational surface behind it.

Submariner: join the networks, route the packets​

Submariner's pitch is the most literal reading of "connect two clusters": make pod IPs in Falkenstein routable from Helsinki and back, over encrypted tunnels. A designated broker holds the membership state, each cluster runs gateway nodes that terminate the tunnels, a route agent programs routes on every node, and Lighthouse implements the Kubernetes Multi-Cluster Services (MCS) API so a ServiceExport in one cluster becomes discoverable in the other.

The concrete demand list for a Hetzner-split CAPI fleet starts with the broker. Some cluster — often a small management cluster, which a CAPI fleet usually has anyway — hosts the Submariner broker namespace, and every workload cluster joins against it. Then each cluster needs gateway nodes: machines with a stable, reachable endpoint that can hold the tunnel. On Hetzner that means a public IP or a reachable private endpoint per gateway, plus firewall rules that let the tunnel protocol through. Cable drivers are pluggable — libreswan IPsec is the default, WireGuard is the modern pick, VXLAN exists for unencrypted overlays you should not run across regions — and the choice is per-deployment, so WireGuard-over-public-internet is the sane default for FSN-to-HEL.

Then comes the gotcha that bites CAPI fleets specifically: overlapping CIDRs. If both regional clusters were stamped from the same cluster template — and of course they were, that is the point of declarative fleet management — they share identical Pod and Service CIDRs. Two pods in two regions can hold the same IP, and no tunnel can route that. Submariner's answer is Globalnet, an egress remapping layer that assigns each cluster a distinct global CIDR and NATs cross-cluster traffic through it. It works, it is proven, and it is a second addressing scheme you now own: global CIDR allocation per cluster, gateway load for the NAT path, and a debugging story where the source IP a Helsinki pod sees is not the IP the Falkenstein pod has.

The failure mode to internalize is blast radius. Once the L3 join is up, every pod can dial every pod across regions unless NetworkPolicy says otherwise — and NetworkPolicy is per-cluster, so your allowlist now lives in two places that must agree. Gateway HA is active/passive failover between gateway nodes, which is robust but means cross-region throughput funnels through one machine at a time. For operator traffic and stateful replication that need true L3 reachability, this is the right shape. For tenant web traffic, it is a bigger merge than most tenants asked for.

Skupper v2: expose services, not networks​

Skupper inverts the model: the two clusters never share a network at all. Each cluster gets a Skupper site — a set of L7 routers — sites connect to each other over ordinary TLS links bootstrapped by token claims, and individual services cross the boundary only when you create them on both sides: a listener on the consuming side, a connector on the providing side. Nothing is reachable across regions except the services you named. There is no VPN, no shared control plane, no CIDR coordination, because at no point do the underlay networks touch.

The demand list is correspondingly small, which is the point. Install the v2 CRDs and controller per cluster (skupper.io/v2alpha1, installable as plain YAML, so it fits a GitOps flow), create a site per cluster, generate a link token on one side and redeem it on the other. The links are outbound TLS connections, so the firewall story is trivial compared to tunnel protocols: no IPsec ports, no WireGuard UDP to negotiate, just the router endpoint. Red Hat ships this line as Red Hat Service Interconnect, which is a reasonable proxy for "someone runs this in production and answers support tickets about it."

The failure modes are L7-shaped. Every cross-region call passes through an app-layer proxy hop, adding latency that matters for chatty service-to-service paths and barely registers for ordinary request/response tenant traffic. Skupper routes what it understands — TCP/UDP service traffic — so anything needing true L3 (custom protocols, pod-to-pod dialing outside declared services, node-level replication) simply does not fit. And the per-service exposure model that makes Skupper safe is also its tax: each service crossing the boundary is a listener/connector pair somebody declares. For a PaaS that already knows its tenants' service graphs, that declaration can be generated. For ad-hoc operator dialing — "SSH to that pod from here" — it is friction by design.

Note the version split: Skupper v1 and v2 sites are not interoperable, v1 stays maintained but gets no significant new features, and all forward motion is on v2. Any new deployment in 2026 should be v2; treat v1 tutorials as history.

Istio ambient multicluster: one mesh, two networks​

The third option arrived at KubeCon EU 2026, when CNCF announced ambient multicluster beta alongside the Gateway API Inference Extension beta: Istio without sidecars, stretched across clusters. Ambient's pitch was always "mesh without per-pod proxies" via node-level ztunnels and optional per-namespace waypoint proxies; multicluster extends that to "mesh without per-pod proxies, across networks," with east-west gateways carrying mTLS between clusters. Istio 1.30 (May 2026) hardened the release further. If your fleet already runs Istio, this is the link with no new component to learn.

But beta means beta, and the documented limitations read like a checklist of things a Hetzner-split fleet must accept explicitly. First, only multi-network, multi-primary topologies are supported: every cluster runs its own istiod, there is no primary-remote shape, and single-network multicluster is untested and possibly broken. Two regions means two networks, so the supported shape happens to match — but you are running N control planes, not one, and each is a full Istio install with ambient enabled.

Second, waypoints are your synchronization problem. Ambient multicluster assumes identically named waypoint deployments in every cluster, and nothing syncs them for you — the docs point at Flux or ArgoCD, which a CAPI fleet likely runs anyway, but the invariant ("same name, same config, every cluster") is yours to enforce. Service scope must be uniform too: only the local cluster's scope config is honored, remote scopes are ignored, so a service scoped differently in FSN and HEL produces traffic behavior neither side intended. And cross-network load balancing has a known skew — requests failing over to a remote network can pile onto a single endpoint through connection multiplexing, tracked upstream, with the same wart present in sidecar mode.

The failure mode is commitment weight. Submariner asks for gateways, Skupper asks for service declarations; ambient multicluster asks for the mesh. You get the richest payoff of the three — SPIFFE workload identity across regions, mesh-wide authorization policy, L7 routing and telemetry that actually understands both sides — but you adopt Istio's control plane, upgrade cadence, and beta limitation list to get a link. That trade is excellent if the mesh was already in your plans and terrible if the link was the only thing you wanted.

The Hetzner wrinkle: your regions are not one network​

Whichever option you pick, two Hetzner facts shape the deployment. The first is that identical CAPI templates make overlapping CIDRs the default, not the edge case. Stamping FSN and HEL from the same Cluster template means the same Pod CIDR twice — which turns Submariner's Globalnet from an optional feature into day-one load-bearing config, while Skupper and Istio sail past the issue entirely (Skupper never merges networks; the mesh routes on identity above pod IPs). If you go the L3 route, budget the global CIDR plan before the first join, not after the first collision.

The second is that inter-region traffic crosses infrastructure you do not control, so encryption is mandatory rather than defense-in-depth. All three options encrypt — WireGuard/IPsec tunnels, TLS router links, mTLS gateways — but they encrypt at different layers with different identity stories. Tunnel encryption proves the clusters trust each other; mesh mTLS proves the workloads do. For tenant traffic crossing FSN-to-HEL, workload identity is the stronger claim, and it is only Istio that gives it to you natively.

Verdict: which layer should a git-push PaaS own?​

Map the pick to the traffic, not the hype:

  • Tenant service-to-service calls across regions — the frontend-in-FSN, API-in-HEL case — belong on Skupper (L7). The per-service allowlist is a feature for multi-tenant traffic: nothing crosses regions except named services, there is no CIDR plan to maintain, and no shared control plane couples your regions' fates. The proxy-hop latency is real but irrelevant next to inter-region RTT, which dominates anyway.
  • True L3 needs — stateful replication, operator pod-to-pod reachability, protocols no proxy understands — belong on Submariner. Accept Globalnet as the price of identical templates, run WireGuard as the cable driver, and write the cross-cluster NetworkPolicy story down before the join, not after the first surprise connection.
  • Fleets already committed to Istio should take ambient multicluster and accept the beta terms: multi-primary only, GitOps-synced waypoints, uniform service scope, eyes open on remote-endpoint load skew. Do not adopt a mesh to get a link; do take the link the mesh now offers if the mesh is already yours.

What not to pick: running two of these at once "for resilience." Overlapping L3 reachability plus L7 routing plus mesh policy is three debugging stories for one packet. One layer, owned deliberately, with the other layers' jobs explicitly declined.

The link decision is also refreshingly orthogonal to the workload roadmap. GPU scheduling, batch queues, inference pools — none of them change which layer joins your regions. Regions get added rarely and links get touched rarely; pick the layer whose failure mode you would rather debug at 3 a.m. from one region while the other is on fire, because that is the only review that counts.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.

Related articles

Run this on infrastructure you own

bex is the open-source, AI-native Render alternative — push a git repo and get a running HTTPS service on your own machines.

Get started with bex