GPU Inference's Hidden Egress Tax: The 22-31% Line Item Nobody Prices In

Run the numbers on a single always-on inference GPU at real production traffic, and data egress lands at 23-31% of the total bill — on AWS, on Azure, and on GCP, independently. That's not a rounding error buried in a footnote; it's roughly a quarter to a third of what a team budgeted for "the GPU" actually going to the pipe the response travels through, not the silicon that generated it. GPU-hours are barely half the story once a model is actually serving user-facing output at volume, and the sticker price on the instance page never mentions the other line item.

Where the 22-31% Number Comes From​

The gap comes from how hyperscalers meter outbound data transfer. All three charge per gigabyte for traffic leaving their network toward the internet, and the rates — while individually modest-sounding — compound fast against a service that's actually answering requests instead of sitting idle:

None of these rates look dangerous on their own — a tenth of a cent short of a dime per gigabyte reads like nothing next to a $3-4/hour GPU. The catch is that the free allotment is tiny and fixed while the GPU keeps running around the clock, so as soon as a service is actually busy, egress stops being a rounding error and starts scaling linearly with the thing the GPU exists to produce: output leaving the building. Independent teardowns pricing identical inference workloads across multiple GPU clouds keep landing on the same finding — egress is the line item that surprises people, because nobody prices it in when they're comparing GPU-hour rates.

The Worked Example, Run Twice​

The honest way to show this isn't one flattering data point — it's the same instance at two traffic levels, because the percentage is traffic-dependent, not fixed. Here's a single always-on A100-class inference GPU (24/7, 730 hours/month) on each provider's real per-GPU rate: AWS's p4d.24xlarge (8×A100, $32.77/hr, ~$4.10/GPU-hour effective), Azure's NC24ads A100 v4 (single A100 80GB, $3.673/hr), and GCP's a2-highgpu-1g (single A100 40GB, $3.673/hr) — three different ways of buying the same GPU class, at three real sticker prices.

Low traffic (500GB/month response egress — a modest RAG assistant, nowhere near saturated):

High traffic (10TB/month response egress — a production chat or agent-serving API actually being used):

At low traffic, egress is genuinely a rounding error — 1-2% of the bill, invisible next to the GPU line. At real production volume, the exact same instance, on the exact same provider, with the exact same rate card, sees egress climb to 23-31% of total spend — matching the range multiple independent 2026 cost teardowns report for production LLM inference bills. The percentage isn't a fixed tax; it's a function of how much the model is actually being used — which means the services egress hurts most are exactly the ones succeeding at their job.

The table also understates the problem for one specific category of workload: an agent-serving layer, not a plain chat endpoint. A single user turn against a tool-calling agent can mean several completions chained together — a planning call, one or more tool-result round-trips, a final synthesis response — and every hop that returns data to the caller is its own slice of egress, even though only the last one is what the user sees. A service fielding agent traffic can rack up several times the egress of an equivalent single-shot chat API at the same request count, which pushes it toward the high end of that 23-31% band faster than the request volume alone would suggest.

The Provider That Zeroed It Out​

Egress isn't a law of physics — it's a pricing choice, and at least one hyperscaler just proved it by dropping the choice entirely. In February 2026, Oracle eliminated all outbound data transfer charges across its full 48-region commercial footprint — internet egress and inter-region transfer both, on every compute shape including its GPU lines. That replaced a prior model (10TB free, then $0.0085/GB) that was already cheaper than AWS, Azure, or GCP, with a flat zero. Whatever else is true about OCI's GPU availability or ecosystem maturity, the egress line on an Oracle inference bill is the one line from the table above that simply doesn't exist.

That's the existence proof worth sitting with: the 22-31% isn't an inherent cost of moving bytes off a GPU. It's the specific pricing decision three vendors made and one vendor unmade.

What Self-Hosting the Serving Layer Actually Removes​

A dedicated GPU box with flat-rate, unmetered bandwidth — the model a self-hosted platform runs on when it provisions its own hardware instead of renting a hyperscaler's metered instance — removes exactly the line item in the right-hand column of both tables above. Not a discount on it, not a higher free tier: the egress meter simply isn't running. At the high-traffic 10TB scenario, that's $861-$1,200/month that stops being billed at all, on top of whatever the compute-per-GB-of-VRAM math already favors (a comparison this list has worked through in detail for Hetzner's dedicated GPU line elsewhere). For an inference or agent-serving layer that's actually busy — which, per the table above, is precisely when egress stops being a rounding error — that's not a marginal saving. It's the difference between a bill where a quarter to a third of the total goes to bandwidth and one where all of it goes to the GPU actually doing the work.

What Doesn't Disappear​

Here's the honest caveat the pitch above leaves out if it stops there: "egress is free" and "self-hosting AI inference is free" are not the same claim, and treating them as interchangeable is where a self-hosting cost pitch overreaches.

The GPU still costs money whether it's busy or not. A flat-rate dedicated box has no scale-to-zero the way a pay-per-token cloud API does — if traffic is bursty or the model sits idle overnight, that idle time is still on the meter, just a different meter than egress. The cost that disappeared was the variable one; the fixed one (the GPU itself) is exactly as fixed as it was on a reserved cloud instance, and a service with genuinely unpredictable, spiky traffic may still find a pay-as-you-go GPU cheaper on the compute line even after losing on egress.

Power and cooling don't vanish — they're just already priced in. On a rented dedicated box, the flat monthly rate already bundles the electricity and the cooling that keeps an A100-class card from thermal-throttling; nobody sends a separate line item for it. But "bundled into the sticker price" isn't the same as "free" — it's the same cost, already amortized into the number a team is comparing against the cloud bill, not a cost that got eliminated by the act of self-hosting.

Running the serving stack is still real ops work. vLLM or TGI, batching and quantization tuning, autoscaling policy, request-level monitoring, the 2am page when a node falls out of the pool — none of that is a line item a cloud API bills separately, but all of it is work a team takes on the moment it stops paying a hyperscaler (or an inference-API vendor) to have already solved it. Zeroing out the egress meter doesn't zero out the engineering hours that keep the serving layer up.

The Narrower, More Honest Claim​

"Egress is free" is a specific, verifiable claim about one metered line item, and the tables above show exactly what it's worth: 1-2% of the bill at low traffic, 23-31% at the volume where a service is actually succeeding. "Self-hosting AI inference is free" is a different, much larger claim that the egress number alone doesn't support — the GPU's fixed cost, the power and cooling already baked into it, and the ops time to run the stack are all still there. The honest pitch for owning the machine the traffic leaves from is the first claim, not the second — and it's a large enough number on its own, at real production volume, not to need the second one to be worth making.

Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own, with no metered egress line hiding behind the compute bill. Star the repo on GitHub or deploy your first inference-serving app today.