Hetzner Unfroze the 50-Second Load Balancer Idle Timeout — Your Build Logs Will Still Die at 15 Seconds

Every platform team that streams build logs has fielded the same bug report, and it is always worded the same way: "the deploy works, but the log window goes dead halfway through." Then a screenshot, always cut off at roughly the same place — somewhere in the quiet stretch where the builder is resolving dependencies and printing nothing.

The app is fine. The build is fine. What died is an idle TCP connection, killed by a timeout in a hop nobody on the team configured, because nobody on the team knew it was there. On April 30, 2026 Hetzner made one of those hops configurable for the first time: HTTP and HTTPS Load Balancer services gained an http.timeout_idle property, letting you move the previously hard-wired 50-second timeout anywhere between 30 and 300 seconds. The Console caught up on June 29. It is a genuinely welcome change — and on a typical Kubernetes-based platform it will fix approximately nothing, because the Hetzner load balancer is usually not the shortest rung on the ladder.

The whole ladder, in one table​

A streaming response passes through four or five timeout enforcers before it reaches a browser. Each one independently decides your stream has been idle too long. The binding constraint is not the one you just tuned — it is the smallest one on the path.

Here is the full ladder for a self-hosted PaaS running on Hetzner and Kubernetes, with the distinction that actually determines what to do about each rung:

Two rules make this table usable, and they are not the same rule:

Idle timeouts are defeated by bytes. For every row marked "idle," the timer resets whenever anything crosses the wire. Your effective idle budget is the minimum across all idle rungs. Raising the Hetzner LB from 50s to 300s while ingress-nginx sits at its default 60s buys you exactly zero seconds.

Total-duration timeouts are not defeated by anything. Envoy Gateway's 15-second default request timeout is a cap on the entire response, not on gaps within it. Heartbeats do not help. Keepalives do not help. A response that never ends will be killed at 15 seconds no matter how chatty it is — which is why, on an Envoy Gateway stack, the streaming endpoint that "cuts out during dependency install" is often cutting out before dependency install even starts, and the 50-second Hetzner timeout was never involved.

Fix the total-duration rung first. Only then does the idle math matter.

Does the new Hetzner knob even apply to you?​

http.timeout_idle is a property of an HTTP/HTTPS service on a Hetzner load balancer. It does not exist on a TCP service.

This matters more than it sounds, because the most common layout for a self-hosted PaaS puts the Hetzner LB in TCP mode: PROXY protocol on, TLS terminated further in at ingress-nginx or Envoy so that cert-manager owns the certificates and per-tenant SNI works. In that topology the load balancer is a dumb L4 forwarder, and the shiny new knob is simply unreachable.

Check before you plan around it:

hcloud load-balancer describe my-lb -o json | jq '.services[] | {protocol, listen_port, http}'

If protocol is tcp, the top rung is not tunable and everything below has to carry the load — which raises the stakes on heartbeats considerably. If it is http or https, you have four ways to set the value:

# hcloud CLI
hcloud load-balancer update-service my-lb --listen-port 443 --http-timeout-idle 300s

# Terraform: hcloud_load_balancer_service
resource "hcloud_load_balancer_service" "app" {
  load_balancer_id = hcloud_load_balancer.main.id
  protocol         = "https"
  http {
    timeout_idle = 300
  }
}

The API path is POST /load_balancers/{id}/actions/update_service. And on Kubernetes there is a fourth, which comes with a caveat worth knowing before you write it into a manifest.

The CCM annotation exists, but has not shipped​

hcloud-cloud-controller-manager defines the annotation:

// LBSvcHTTPTimeoutIdle specifies the idle timeout for the client and
// server side. Must be between 30s and 300s.
//
// Type: duration
LBSvcHTTPTimeoutIdle Name = "load-balancer.hetzner.cloud/http-timeout-idle"

It parses it and threads it into both the create and update paths. But it is on main only — it is absent from every tagged release through v1.34.0 (July 17, 2026). If you add load-balancer.hetzner.cloud/http-timeout-idle: 300s to a Service today, a released CCM will ignore it silently, which is the worst possible failure mode: the manifest looks correct in review and does nothing in production.

Until it ships, set the value out of band via CLI, Terraform, or the API. Because today's released CCM has no concept of the field, it does not send it during reconciliation and your out-of-band value should survive — but confirm that by forcing a reconcile (touch an unrelated load-balancer.hetzner.cloud/* annotation) and re-describing the service. Once the annotation lands in a release, move to it and delete the out-of-band step; two systems writing the same field is a drift bug waiting for a bad afternoon.

Why it reads as a flaky app instead of a timeout​

The reason these bugs survive multiple debugging sessions is that the two most common hops fail in visibly different ways, and one of them fails invisibly.

ingress-nginx is loud. When proxy_read_timeout fires, NGINX logs upstream timed out and returns a 504 to the client. You get a status code, a log line, and something to grep for.

The Hetzner load balancer is quiet. When the idle timeout fires it closes the connection. To the browser, a closed connection on a chunked response is indistinguishable from the server having finished: no error, no status code, just end-of-stream.

And then EventSource does what it is specified to do — reconnects automatically after a few seconds. If your log endpoint replays from the beginning on reconnect, the user sees the first 200 lines again. If it resumes from a cursor, they see a silent gap. Either way the symptom presents as "the log viewer is buggy," and the team spends a sprint in the frontend.

To find the guilty rung, take the browser out of it and time the death:

curl -N -s -o /dev/null -w '\nclosed after %{time_total}s (http %{http_code})\n' \
  https://app.example.com/v1/deploys/abc123/logs

Read the result against the ladder:

• ~15s, no data, HTTP 200 then truncation — Envoy Gateway's default request timeout. A total-duration cap; heartbeats will not save you.
• ~50s — the Hetzner LB at its factory default, on an HTTP/HTTPS service.
• ~60s with a 504 — ingress-nginx proxy_read_timeout.
• ~5 min — Envoy's stream_idle_timeout.
• Dies only for some users, on some networks — a middlebox you do not own. No amount of server-side configuration fixes this one.

That last case is the one that turns "just raise the timeouts" from a fix into a coping strategy.

Why 300 seconds still is not the answer​

You can raise every rung you control to its ceiling and still lose connections. Three reasons:

You do not own the whole path. Corporate egress proxies, home-router NAT tables, and mobile carrier gateways all have idle timeouts, commonly in the 60–120 second range, and none of them read your Terraform. A silent stream is a dead stream on those networks regardless of what Hetzner allows.

A long timeout is not liveness detection. If a backend hangs, a 300-second idle timeout means the user stares at a frozen log pane for five minutes before anything notices. Heartbeats let both ends detect death in seconds.

Ceilings are a vendor's choice, not yours. 300 seconds is today's cap. Designing your protocol so that it depends on that number makes your product's correctness a function of someone else's changelog.

The correct fix is to make the stream never idle. Derive the interval from the ladder:

heartbeat interval ≤ ½ × (smallest idle timeout on the path)

Worked example, in order:

1. Disable the total-duration rung. On Envoy Gateway, set the HTTPRoute's timeouts.request to 0s for the log-streaming route. Until this is done, the rest of the math is irrelevant.
2. Find the smallest remaining idle rung. LB raised to 300s, Envoy stream_idle_timeout at 5 min, ingress-nginx still at its default 60s. Budget: 60 seconds.
3. Halve it, then leave margin for the networks you cannot see. 60 ÷ 2 = 30s; a 20–25 second interval is comfortable and survives most middleboxes.

For server-sent events, a heartbeat is a comment frame — a line starting with a colon, which EventSource ignores but which is unambiguously bytes on the wire:

: ping
 

For WebSockets, use protocol-level ping/pong frames rather than application-level messages. Proxies and load balancers recognize control frames as activity and reset their idle timers, and you get dead-peer detection for free.

Setting it end to end​

Pick the stack you actually run. A configuration that is half one and half the other is the usual reason a "fix" does not take.

ingress-nginx stack

# 1. Service fronted by the Hetzner LB (annotation once it ships;
#    until then set http.timeout_idle via CLI/Terraform)
apiVersion: v1
kind: Service
metadata:
  annotations:
    load-balancer.hetzner.cloud/protocol: https
    load-balancer.hetzner.cloud/http-timeout-idle: 300s

# 2. The streaming route
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-buffering: "off"

proxy-buffering: off is not optional. With buffering on, NGINX may hold your heartbeats and log lines until a buffer fills — the stream looks idle to everything downstream and to the user, even though the backend is writing.

Envoy Gateway stack

# 1. Kill the 15s total-duration cap on the streaming route only
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
spec:
  rules:
    - matches:
        - path: { type: PathPrefix, value: /v1/deploys }
      timeouts:
        request: 0s

# 2. Raise the listener stream idle timeout
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
spec:
  timeout:
    http:
      streamIdleTimeout: 1h

Two sharp edges here. streamIdleTimeout in ClientTrafficPolicy is listener-scoped — it applies to every route on that listener, not just the streaming one. Envoy Gateway has no per-route equivalent in BackendTrafficPolicy yet (issue #8454). And if anything sets a route-level idle_timeout, it silently overrides stream_idle_timeout for that route — a route-level tweak intended to tighten one endpoint can quietly loosen or shorten another.

Then, on both stacks, the part that makes the rest robust:

// SSE heartbeat — 20s, comfortably under a 60s budget
const heartbeat = setInterval(() => res.write(": ping\n\n"), 20_000);
res.on("close", () => clearInterval(heartbeat));

The shortest rung wins​

The Hetzner change is a real improvement, and if you terminate TLS at the load balancer you should take the 300 seconds. But the lesson generalizes past one vendor's changelog: a request path's timeout behavior is determined by its shortest rung, and every layer you add brings its own defaults with it. Adopting Gateway API introduces a 15-second total-duration cap that ingress-nginx never had. Adding a managed load balancer introduces a timeout your bare-metal setup did not have. Nobody chose these numbers for your workload; they arrived as defaults.

So the durable practice is not "raise the timeout." It is: enumerate the rungs, label each as idle or total, disable the total ones on streaming routes, take the minimum of the idle ones, and heartbeat at half of it. Then treat that enumeration as configuration you own, not folklore in someone's head — because the next person to add a service mesh or swap an ingress controller will insert a new rung, and the only thing that catches it is knowing the ladder was ever there.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with live build logs that stream all the way to your browser. Star the repo on GitHub or deploy your first app today.