Your Kubernetes Autoscaler Can't See Your Queue: Building the Exporter That Fixes It

A worker pod calling a slow third-party API sits at 5% CPU while its job queue backs up to a thousand pending messages. The Horizontal Pod Autoscaler watches CPU. It sees 5% utilization, decides everything is fine, and never scales past minReplicas: 1. The backlog keeps growing. Nothing in the autoscaler's view of the world knows it exists.

This is the failure mode a July 14, 2026 Kubernetes blog post by contributor Victor David Effiok sets out to fix, and it's worth being precise about why it happens: CPU and memory percentages are a proxy for load, not load itself. That proxy holds for CPU-bound request handlers. It breaks completely for I/O-bound async workers — anything blocked waiting on a network call, a downstream API, or a slow database query spends most of its time doing nothing measurable while the actual signal that matters, the backlog, climbs unseen. The fix is a custom metrics exporter: a small HTTP server that turns your application's own state — queue depth, job duration, open WebSocket count — into something Kubernetes' autoscaler can actually read. Here's exactly how that exporter gets built and wired in, and what it means for a self-hosted PaaS's default autoscaling.

This Isn't a Kubernetes-Only Problem​

Before getting into the exporter itself, it's worth establishing that this gap isn't a Kubernetes quirk — it's a property of CPU/memory-based autoscaling in general, and platforms without Kubernetes hit it just as hard. Heroku, Render, and Railway all autoscale worker dynos/services on CPU and memory by default, and all three have spawned the same third-party fix: Judoscale, an add-on built specifically to autoscale background workers on job queue time — how long a job actually waits before a worker picks it up — instead of CPU or memory. Judoscale's own pitch is blunt about why it exists: worker dynos can sit at full CPU allocation while idle, or process a queue spike without any proportional resource increase, because the resource metrics and the actual backlog are only loosely correlated for I/O-bound work.

That's the same gap the Kubernetes exporter closes, just solved with a bolt-on product instead of a native signal. A platform whose autoscaling is CPU/memory-only isn't making a mistake — CPU-based HPA is the correct choice for stateless, CPU-bound request/response services, where utilization and load really do move together. It's specifically async worker/queue topologies where the proxy fails, and that's exactly the workload shape a git-push PaaS's tenants disproportionately run: webhook receivers, background job processors, batch pipelines.

Building the Exporter: Metrics, Code, and the /metrics Endpoint​

An exporter's entire job is narrow: expose application state as plain text on a /metrics endpoint, on a fixed schedule Prometheus can scrape. Prometheus's data model has three metric types that matter here:

Those three names are the actual examples the Kubernetes blog post uses, and they map directly onto the failure mode above: worker_queue_depth is the backlog CPU can't see, and worker_job_duration_seconds is what lets you tell "queue is growing because load increased" apart from "queue is growing because jobs got slower" — two different problems with two different fixes.

The Go implementation is a handful of lines against the official Prometheus client library:

go mod init example.com/my-exporter
go get github.com/prometheus/client_golang/prometheus
go get github.com/prometheus/client_golang/prometheus/promhttp

package main
 
import (
    "log"
    "net/http"
 
    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promhttp"
)
 
var (
    jobsProcessed = prometheus.NewCounterVec(
        prometheus.CounterOpts{
            Name: "worker_jobs_processed_total",
            Help: "Total number of jobs processed, partitioned by status.",
        },
        []string{"status"},
    )
 
    queueDepth = prometheus.NewGauge(prometheus.GaugeOpts{
        Name: "worker_queue_depth",
        Help: "Current number of jobs waiting in the queue.",
    })
 
    jobDuration = prometheus.NewHistogram(prometheus.HistogramOpts{
        Name: "worker_job_duration_seconds",
        Help: "Time spent processing jobs, in seconds.",
    })
)
 
func init() {
    prometheus.MustRegister(jobsProcessed)
    prometheus.MustRegister(queueDepth)
    prometheus.MustRegister(jobDuration)
}
 
func main() {
    http.Handle("/metrics", promhttp.Handler())
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Registering a metric doesn't require an observation to exist first — declaring queueDepth makes it show up on /metrics at zero the moment the process starts, which matters because an HPA querying a metric that's never been registered gets no data at all, not a zero.

Wiring It Into the Cluster​

The exporter is useless to the autoscaler until three more pieces connect it to the custom metrics API. First, the workload needs a Service exposing the metrics port so Prometheus has something to scrape:

apiVersion: v1
kind: Service
metadata:
  name: my-worker-metrics
spec:
  selector:
    app: my-worker
  ports:
  - port: 8080
    targetPort: 8080
    name: metrics

Second, if the cluster runs the Prometheus Operator, a ServiceMonitor tells Prometheus to scrape it on a schedule:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: my-worker-metrics
spec:
  selector:
    matchLabels:
      app: my-worker
  endpoints:
  - port: metrics
    interval: 30s

Third — and this is the step that's easy to skip and then wonder why the HPA sees nothing — Prometheus holding the metric isn't the same as Kubernetes' autoscaling API being able to query it. The prometheus-adapter is what implements custom.metrics.k8s.io, translating a PromQL query into something the HPA controller can call directly. A rule maps worker_queue_depth into the custom metrics API under a shorter name:

apiVersion: v1
kind: ConfigMap
metadata:
  name: adapter-config
  namespace: custom-metrics
data:
  config.yaml: |
    rules:
    - seriesQuery: 'worker_queue_depth'
      resources:
        overrides:
          namespace: {resource: "namespace"}
          pod: {resource: "pod"}
      name:
        matches: "^worker_queue_depth"
        as: "queue_depth"
      metricsQuery: 'worker_queue_depth{<<.LabelMatchers>>}'

Once that's applied, kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1" | jq . should list queue_depth — that's the confirmation the whole pipeline (exporter → scrape → adapter) is actually connected before touching the HPA at all.

Finally, the HorizontalPodAutoscaler itself. Worth being precise here: autoscaling/v2 (stable since Kubernetes 1.23, and the only version that should be used going forward) nests the metric name and target under metric:/target: rather than the flat metricName/targetAverageValue fields from the long-deprecated v2beta1 API:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-worker
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-worker
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Pods
    pods:
      metric:
        name: queue_depth
      target:
        type: AverageValue
        averageValue: "30"

That config scales my-worker to keep queue depth at roughly 30 pending jobs per pod — 60 pending jobs pushes it toward 2 replicas, 300 toward the 10-replica ceiling, all without CPU or memory ever entering the decision.

Same Deployment, Two Autoscalers​

Put the two approaches side by side on the exact worker from the hook — an I/O-bound webhook processor blocked on a slow downstream API:

That last row matters: this isn't an argument that CPU/memory autoscaling is broken everywhere. It's the right tool for CPU-bound services where utilization and load move together. The exporter only earns its keep on the workload shape where that assumption fails — async workers, queue consumers, batch jobs — and a platform serving both shapes needs both tools, not a blanket replacement.

One adjacent tool worth placing precisely: KEDA solves a different half of this problem. KEDA's job is the 0↔1 transition — taking a Deployment to true zero replicas and waking it from an event source — and above 1 replica it hands off to a standard HPA underneath. KEDA still needs a custom metric to scale on anything HPA-native signals can't see, which means an exporter like the one above isn't a KEDA alternative — it's very often the exact metric source KEDA's ScaledObject polls once a workload is already running.

What a Golden Path Looks Like on a Cluster API Fleet​

None of the four manifests above are exotic, but they are four separate files a tenant has to write correctly, in order, before their worker's autoscaling does anything but CPU-react. On a Cluster API–managed fleet running many tenants' worker services side by side, that's a repeated cost every team pays alone — and a common way for it to silently not work, since a missing ServiceMonitor or a wrong seriesQuery produces no error, just an HPA that never scales.

A self-hosted PaaS built on Cluster API is positioned to collapse that into an opt-in template instead of leaving each tenant to hand-roll it:

• A scaffolded exporter template — a worker app type that ships the client_golang/promhttp boilerplate above pre-wired, so a tenant adds queueDepth.Set(n) calls to their own job-processing code and gets a working /metrics endpoint for free.
• Auto-generated ServiceMonitor and adapter rule per tenant namespace — templated off the tenant's own metric names at deploy time, scoped by the same namespace-per-tenant RBAC boundary the platform already enforces for everything else, so one tenant's queue_depth rule can't leak into another's HPA.
• An HPA stanza in the platform's own deploy config — something as small as autoscale: { metric: queue_depth, target: 30 } in a bex.yml, expanding to the full autoscaling/v2 manifest above at apply time, instead of a tenant copy-pasting YAML from a blog post.

The payoff isn't cosmetic. It's the difference between a worker tenant discovering, after an incident, that their autoscaler has been watching the wrong number the whole time, and that tenant getting queue-aware scaling as a checkbox the same way they already get a TLS certificate or a custom domain.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.