CPU Lies: What a Git-Push PaaS Should Autoscale Tenants On Instead

Your Node service is melting. Event loop lag is through the roof, p95 response time just tripled, users are retrying — and the autoscaler is doing nothing, because CPU sits at a comfortable 35%. The pods are not compute-bound. They are waiting: on a downstream API, on a full connection pool, on a queue nobody is draining. CPU never sees any of that, which is why every platform operator eventually learns the same lesson — CPU is a lagging, often misleading autoscaling signal for web workloads, and the metrics that actually predict saturation have to be exported and plumbed into the scheduler by hand.

Here is the verdict up front: a sane default autoscaling spec for a git-push web service scales on two HPA-grade signals, watches two more as guardrails, and leaves CPU as the backstop it always should have been:

The rest of this post builds the pipeline that makes the first two rows real: export the metric, scrape it, serve it through a metrics API, and let the HorizontalPodAutoscaler act on it — plus the two decisions every multi-tenant platform gets wrong the first time (adapter vs. KEDA, and how to keep one tenant's metrics from breaking autoscaling for everyone), and what Render and Railway actually do today.

What the July Kubernetes exporter guide actually builds​

The Kubernetes blog's July 14, 2026 walkthrough by Victor David Effiok is deliberately boring in the best way: an exporter is a small HTTP server with one job — expose application state as text on /metrics so Prometheus can scrape it. When you control the application code you can embed the client library directly; a standalone exporter earns its keep when the data source is external to the app or you do not own the code.

The guide's core decision, and the one worth stealing verbatim, is which Prometheus type each signal gets. Counters only increase (requests served, jobs processed — never use one for a value that can fall). Gauges snapshot a value that rises and falls freely: queue depth, active connections, in-flight requests. Histograms record distributions so you can compute percentiles rather than averages. The example is a job processor exposing worker_jobs_processed_total, worker_queue_depth, and worker_job_duration_seconds, following the <namespace>_<name>_<unit> snake_case convention. Names are a contract here — the adapter rule you write in three sections' time matches on them — so pick them once and treat renames as breaking changes.

The operational details that matter for a platform, not just a demo: the polling interval that refreshes a gauge must be shorter than Prometheus's scrape interval (five seconds against a typical fifteen-second scrape, in the guide's example) so every scrape sees a fresh value. The container ships distroless and non-root so it passes a default security policy without extra work.

And the Deployment plus Service live in the monitoring namespace with the port literally named metrics, because the ServiceMonitor in the next leg references that name. None of this is glamorous. All of it is the difference between a metrics pipeline and a metrics outage with extra steps.

The leg everyone skips: from Prometheus to the HPA​

Scraped metrics do not reach the autoscaler by themselves. The HPA in autoscaling/v2 speaks five metric types — Resource, ContainerResource, Pods, Object, and External — and everything that is not CPU or memory arrives through one of two API surfaces: custom.metrics.k8s.io for pod- or object-associated series, external.metrics.k8s.io for signals that live outside any Kubernetes object (a cloud queue length, requests-per-second from the load balancer). Something has to serve those APIs. That something is historically the Prometheus Adapter: an aggregated APIService that evaluates a PromQL query per HPA poll and presents the number as if it were a native metric.

A minimal adapter rule for the queue-depth gauge looks like this:

rules:
- seriesQuery: 'worker_queue_depth{namespace!="",pod!=""}'
  resources:
    overrides:
      namespace: {resource: "namespace"}
      pod: {resource: "pod"}
  name:
    matches: "^(.*)_depth"
    as: "${1}_per_pod"
  metricsQuery: 'avg(worker_queue_depth) by (namespace, pod)'

And the HPA that consumes it:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: {name: worker, namespace: tenant-acme}
spec:
  scaleTargetRef: {apiVersion: apps/v1, kind: Deployment, name: worker}
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Pods
    pods:
      metric: {name: worker_queue_per_pod}
      target: {type: AverageValue, averageValue: "25"}
  behavior:
    scaleDown: {stabilizationWindowSeconds: 300}
    scaleUp: {stabilizationWindowSeconds: 60}

Two things in that YAML deserve more attention than they usually get. First, the HPA's replica math — roughly, desired replicas equals current replicas times current metric value over target — assumes the metric divides linearly across pods. Requests-in-flight per pod and queue depth per replica satisfy that assumption: double the pods, halve the per-pod share.

This is exactly why p95 latency sits in the guardrail row of the opening table and not the target row. Doubling pods does not halve your p99; latency follows the slowest dependency, the worst shard, the coldest cache. Pointing HPA's proportional controller at a non-linear signal buys oscillation, not capacity. Scale on the divisible quantity; alert on the latency it protects.

Second, targets have sensitivity and it is linear too: set the per-pod target 2x too high and you run half the replicas you needed until latency pages somebody; set it 2x too low and you pay for twice the fleet. There is no auto-tuning here.

The honest default is to ship a per-framework starting point (tens of in-flight requests for a Node service, single digits for a Python sync worker), expose the target as a tenant-tunable field, and graph the metric against the target so the tenant can see the controller's reasoning. The behavior block's stabilization windows are the other half of the same discipline: 60 seconds up, 300 down is a sane default that survives deploys without flapping.

Prometheus-adapter vs. KEDA is a real fork — pick per workload shape​

Every team building this pipeline hits the same fork, and both tines are load-bearing:

The 1.37 update genuinely moves the needle: HPA scale-to-zero reaching beta, enabled by default removes the single strongest reason teams adopted KEDA — idle queue consumers and GPU workloads can now sleep at zero replicas and wake on an external metric without any add-on. The Kubernetes project names those two workload shapes explicitly, and they are exactly a PaaS's background-worker story.

The recommendation that falls out is boring and therefore correct: ship HPA plus the adapter as the default path for HTTP services. The metric — in-flight requests — already lives in Prometheus, the wiring is one rule per signal, and there is no second operator to patch.

Reach for KEDA the moment the signal lives outside Prometheus — a managed queue, a cloud pub/sub backlog, a cron-shaped schedule — or when a tenant asks for scale-to-zero on a workload type your 1.37 upgrade has not covered yet. Note the deprecation weather too: the Prometheus Adapter is widely described as heading for eventual deprecation in favor of KEDA-style paths, so write your adapter rules as generated config from a small schema now, and switching costs stay small later.

The multi-tenant parts: namespacing, cardinality, and protecting the metrics path​

A single-tenant exporter demo ends where a platform's real work begins. Three problems, in the order they will page you:

Per-tenant namespacing. Every exported series must carry the tenant identity as a label (tenant, or equivalently namespace when tenants map to namespaces), and every adapter rule must slice by it — avg(...) by (namespace, pod), never a bare global average, or one tenant's traffic spike scales everybody's fleet. Metric names stay global and stable (they are the platform's API); tenant identity travels exclusively in labels. Enforce it in admission: reject exporter configs and ServiceMonitors missing the tenant label the way you would reject a Deployment without resource limits.

Cardinality budgets. Labels are the Glorious foot-gun of this whole design: a user_id or request_path label on a per-request counter creates a new time series per value, and Prometheus pays memory per series. A tenant who instruments http_requests_total{path="/user/12345"} can OOM your monitoring stack faster than any traffic spike. Cap it three ways: document an allowlist of label keys for tenant exporters, set per-tenant active-series limits at the Prometheus remote-write or agent layer, and drop high-cardinality labels with relabeling rules before ingestion. Cardinality is the noisy-neighbor vector nobody budgets for until it fires.

Protect the metrics path itself. Your autoscaling loop is now a dependency chain — exporter, Prometheus, adapter, HPA controller — and a tenant can break it for everyone by flooding any link. Give the monitoring namespace scheduling priority and resource guarantees that tenant workloads cannot preempt.

Scrape platform exporters on a separate interval or Prometheus instance from tenant-appointed scrape targets, so a pathological /metrics endpoint delays its owner's graphs, not the fleet's scaling decisions. And alert on pipeline health (scrape failures, adapter latency, HPA "unable to get metric" events) with the same severity as a node outage. An autoscaler that cannot see is worse than no autoscaler: it holds the last replica count while the world changes, confidently.

What Render and Railway actually do — imitate this, improve on that​

Ground the advice against the two platforms your migrating users know:

Render does threshold-based horizontal autoscaling on CPU and memory: set minInstances/maxInstances plus targetCPUPercent/targetMemoryPercent in render.yaml or the dashboard, and Render adds instances when either signal exceeds its target — computing the scale decision per metric and applying the larger, more aggressive count — with cooldowns around scale events and a floor of one instance (no scale-to-zero). Imitate: declarative min/max plus targets in config, per-metric evaluation with max-wins, cooldowns before re-firing. Improve: the signal set itself. Render's own model proves tenants understand "target X, between N and M replicas" — keep that UX exactly, but let X be in-flight requests or queue depth with CPU/mem as the backstop row, which is precisely the table at the top of this post.

Railway takes the other fork: vertical autoscaling plus manual horizontal replicas, with no threshold-based horizontal autoscaling that reacts to live load. That is a defensible simplicity choice for a developer-experience-first platform, and it is also the gap: a traffic spike on Railway waits for a human (or an external controller watching from outside) to add replicas. Imitate: the zero-config default — most tenants should never touch autoscaling settings and still survive a spike. Improve: everything else. A platform that already runs the exporter-to-HPA pipeline can offer Railway-simple defaults (preset per-framework targets, the 60/300 stabilization windows baked in) while actually reacting to load, which is the one comparison row where "we do what Railway does, automatically" is honest.

Neither platform exposes custom-metric autoscaling to tenants today. That absence is the opening: the first self-hosted PaaS that ships "set a target on the metric your framework already exports" as a three-line config block — with the guardrails from the previous section invisible underneath — wins the migration argument against both, because it offers something neither incumbent sells at any price.

The default spec, written down​

Concretely, here is what a git-push web service should get on day one without asking: HPA on in-flight-requests-per-pod (framework-sane default target, tenant-tunable), HPA on queue-depth-per-replica for anything with a worker tier, p95 latency and pool saturation as paged guardrails rather than scaling targets, minReplicas from the tenant's availability ask with zero allowed on 1.37+ for idle-eligible workers, 60-second scale-up and 300-second scale-down stabilization, and the whole loop namespaced, cardinality-capped, and priority-scheduled per the previous section. CPU and memory stay in the metric list — as the backstop that catches the workload shape nobody predicted, not as the primary signal for the shapes everyone has.

That is a weekend of YAML and one small exporter library away from any Cluster API fleet — and it is the difference between an autoscaler that watches the load your tenants actually feel and one that watches a percentage on a dashboard while the event loop burns.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.