Your Queue Is Retrying Itself Slow: What OpenFaaS Adaptive Concurrency Fixes About Async Batch Work on Kubernetes

Adding a queue to a batch workload is supposed to make it faster. For one common OpenFaaS setup, it was making it roughly twice as slow as it needed to be — not because the queue was slow, but because the dispatcher was too eager. It fired async invocations at functions as fast as possible, collected a storm of 429 Too Many Requests rejections, and then burned the saved time sitting in exponential back-off.

OpenFaaS's April 2026 queue-worker update fixes this with adaptive concurrency: instead of dispatching greedily and retrying, the queue-worker learns how much work each function can actually accept and throttles itself to match. On the project's own benchmark — the sleep function, max_inflight of 5, up to 10 replicas, one async batch — the same work finished about 50% faster, with the vast majority of requests succeeding on the first attempt.

That number deserves a skeptical read, and then a serious one. The skeptical read: it is a vendor benchmark on a synthetic function. The serious one: the failure mode it removes — retry storms against concurrency-limited functions — is one of the most common ways self-hosted async workloads waste capacity, and the fix requires no per-function tuning. If you run batch work (PDF rendering, inference, transcoding, ETL, webhook fan-out) on Kubernetes, this is worth understanding whether or not you run OpenFaaS. And if you operate a PaaS, it sharpens a roadmap question: is serverless functions a workload type to host, or a primitive to build?

How async invocation actually works in OpenFaaS​

Every OpenFaaS function can be called two ways, and the difference matters more than it looks.

Synchronous is the default: the caller sends an HTTP request, the gateway proxies it to the function, and the caller waits for the response. Simple — unless the function runs five minutes, in which case the caller waits five minutes.

Asynchronous goes through a queue. The caller POSTs to /async-function/<name> and gets back 202 Accepted with an X-Call-Id header within milliseconds. The gateway serializes the request onto a NATS JetStream queue; the queue-worker subscribes, pulls messages off, and invokes the function. Pass an X-Callback-Url header and the result gets POSTed there when the work finishes. Think of it as a hybrid of a batch-job queue and deferred execution: submit the job, optionally subscribe to the result, go do something else.

This shape is ideal for exactly the workloads that embarrass a request/response platform: long-running jobs, batch processing, webhooks with tight response-time contracts where the work itself is slow, and fan-out pipelines. The caller never blocks; the queue absorbs the burst; the functions drain it. That is the theory. The practice had a dispatch problem.

Why the greedy dispatcher generated a retry storm​

By default the queue-worker dispatches greedily: pull messages, send them to the function as fast as possible. For unconstrained functions this is fine and widely used in production. It breaks down the moment a function has a real concurrency limit — which serious batch functions almost always do.

The limit is declared with max_inflight: set it to 5 and each replica accepts 5 concurrent requests; the 6th gets a 429. Sensible limits are small. Headless-Chrome PDF rendering tops out at 1–2 browsers per replica. A GPU-bound inference function runs one inference at a time (max_inflight=1). Video transcoding and ETL steps are bounded by CPU or memory per job. So a batch of hundreds of async invocations arrives, the first few per replica succeed, and everything else is rejected — then retried with exponential back-off, rejected again as autoscaling slowly adds replicas, and eventually cleared. During the whole ramp-up, a large share of requests are retried at least once.

Two costs hide inside that pattern. The obvious one is time: every retry cycle parks work in back-off instead of executing it. The subtle one is phantom load: the 429 churn inflates the metrics the autoscaler reads, and in the project's tests the greedy approach actually drove replica counts higher — scaling against retry noise rather than real demand. You pay twice: slower batches on more replicas than the work needed. Operators compensated with careful per-function retry tuning (maxRetryWait, initialRetryWait, maxRetryAttempts), which is exactly the kind of per-workload knob-tuning that does not survive contact with a multi-tenant platform.

The fix: a dispatcher that learns capacity​

Adaptive concurrency flips the strategy. Instead of dispatching at full speed and managing rejections, the queue-worker probes for each function's real capacity and holds messages in the queue until the function can accept them. The algorithm is a feedback loop:

1. Start low. The queue-worker begins with a concurrency limit near zero per function and grows it incrementally from real responses.
2. Increase on success. Successful responses raise the limit; a sustained rejection-free stretch raises it more aggressively.
3. Back off on rejection. Consecutive 429s cut the limit with a safety margin below the discovered ceiling, so it stops hammering the edge.
4. Probe the backlog. The worker periodically checks for queued work and proactively raises the limit to fill available capacity — no idle replicas while messages wait.
5. Track replica changes. As the autoscaler adds or removes replicas, acceptable throughput changes; the success/failure signal picks that up automatically.

The side-by-side result on the identical batch: roughly 50% faster completion, a collapsed 429 rate, a smooth constant inflight-load curve instead of burst-and-retry sawteeth, and lower replica usage. The mechanism behind the speedup is almost embarrassingly simple — fewer retries means less cumulative back-off time — which is precisely why it generalizes beyond the benchmark. Any queue-plus-limited-worker system pays the same tax; OpenFaaS just stopped paying it by default.

Enabling it is unglamorous, which is a compliment. It ships on by default in the JetStream queue-worker's function mode (per-function NATS consumers), works under any scaling mode — capacity, queue-depth, or RPS — and needs no per-function tuning beyond the max_inflight declaration you should already have. The canonical setup from the release post:

faas-cli store deploy sleep \
  --label com.openfaas.scale.max=10 \
  --label com.openfaas.scale.target=5 \
  --label com.openfaas.scale.type=capacity \
  --label com.openfaas.scale.target-proportion=0.9 \
  --env max_inflight=5

Then fire the batch and watch the queue-worker's Grafana dashboard — queue depth draining steadily while inflight climbs in step with replicas, no spikes, no idle gaps:

hey -m POST -n 500 -c 4 \
  http://127.0.0.1:8080/async-function/sleep

To revert to greedy dispatch, one Helm value does it (jetstreamQueueWorker.adaptiveConcurrency: false) — a small mercy that tells you the maintainers expect some exotic workload to prefer the old behavior.

Where it matters: known limits and unknown ones​

The feature covers two distinct cases, and the second is the more interesting one.

Functions with a known concurrency limit are the obvious win: PDF generation, single-inference GPU functions, transcoding, ETL. The project's own worked example is PDF rendering — a 600-page batch through headless Chrome at max_inflight 1–2 per replica, where greedy dispatch meant flooding, mass 429s, and a tuning session over retry parameters before every large run. Adaptive concurrency learns "one browser per replica" on its own and the limit rises automatically as replicas scale. The tuning session disappears.

Functions with variable upstream capacity need no max_inflight at all. A function backed by a loaded database, a rate-limited third-party API, or an overloaded shared microservice can simply return 429 itself as back-pressure — and the queue-worker treats it the same way: slow down, wait, probe for recovery, climb back when the upstream heals. This is the deeper insight buried in the release: the 429 contract turns any downstream bottleneck, including ones outside Kubernetes entirely, into a signal the dispatcher already understands. No sidecar, no custom rate-limiter, no circuit-breaker library — just a status code with agreed meaning.

One honest scoping note for self-hosters: adaptive concurrency rides on the JetStream queue-worker's function mode, which is commercial OpenFaaS (Pro/Enterprise) territory, not Community Edition. CE operators still get async dispatch and queue-depth-based scaling, but the learning dispatcher is part of what the license buys. Budget accordingly: if your batch fleet lives on CE's greedy dispatch, the retry-tuning burden the release eliminates is still yours. That is not a criticism — it is the actual price tag on the 50%, and it belongs in any build-vs-buy accounting next to the engineering hours of hand-rolled concurrency control.

Should a git-push PaaS host this or build it?​

Here is the roadmap question this post set out to answer: is OpenFaaS a complementary workload type for a container-per-app PaaS to run as a tenant app, or a gap in the platform's own primitives worth closing directly? After the research, the answer splits cleanly by workload shape.

Host it. Async batch and event-driven handlers — scale-to-zero webhooks, per-invocation jobs, fan-out pipelines — are a genuinely different scheduling shape from long-running git-push services. They want queue-depth autoscaling, 202-and-callback semantics, and retry/backoff policy per function; a platform whose autoscaler thinks in CPU/RPS per Deployment will re-derive all of that badly. OpenFaaS installs via Helm onto the same Cluster API fleet, functions are just OCI images, and the queue-worker plus NATS JetStream is ordinary stateful workload the fleet already knows how to run. For the tenant who needs "run this 600-page render batch every night," pointing at a hosted function namespace beats building a FaaS control plane.

The Knative comparison sharpens the point rather than muddying it. Knative Serving answers "scale HTTP to zero" with request-driven autoscaling, revisions, and traffic splitting; Knative Eventing routes CloudEvents over pluggable brokers. OpenFaaS async answers a narrower question — "absorb a batch without the caller waiting" — with queue-native semantics (persistent JetStream backlog, per-function consumers, callback delivery) that fit batch processing more naturally than request-driven scaling ever will. They overlap at the edges; they are not substitutes. A platform choosing one "serverless story" should pick by the workload: spiky HTTP services lean Knative, queued batch leans OpenFaaS.

Don't rebuild it. The failure mode this release fixes is evidence for restraint, not ambition. Greedy dispatch plus retry storms is exactly the bug a platform team writes for itself the first time it bolts a queue onto its own autoscaler — OpenFaaS needed years of production feedback plus a commercial queue-worker rewrite to learn the feedback loop. A git-push PaaS has no business re-learning it. The platform's job is the layer underneath: GPU/CPU node pools the functions schedule onto, per-tenant routing and TLS in front of the gateway, and observable queue depth per tenant. Own the fleet; rent the dispatcher.

There is one exception worth naming: if agent-operated workloads arrive — AI agents firing deployment jobs, eval batches, and sandbox provisions as events — the platform will eventually want first-class async primitives with machine-readable state (queue depth, inflight, per-job status) behind its own API, not just a Helm-installed tenant app. That is a real roadmap dependency, but it is a reason to expose OpenFaaS-style semantics through the platform API later, not to reimplement the dispatcher now.

The broader lesson travels even further than OpenFaaS. Every team running limited workers behind a queue — Sidekiq, Celery, SQS consumers, custom NATS subscribers — pays some version of the greedy tax: dispatch fast, reject, back off, repeat. The fix is always the same shape: close the loop, learn capacity from responses, hold work instead of retrying it. OpenFaaS just shipped the reference implementation for Kubernetes. Steal the pattern even if you never install the chart.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.