Dora NodaA reasoning model roughly as capable as o3-mini now runs on a graphics card you can buy used for the price of a dinner for two — and OpenAI licensed the weights under Apache 2.0, so nothing about serving it phones home. The question is no longer can you self-host frontier-adjacent inference, but whether the math beats paying per token.
The short answer
Here is the verdict up front, with receipts below. If you compare like-for-like — the same open weights served on your hardware versus rented from a serverless inference provider at roughly $0.05 per million input tokens and $0.20 per million output tokens for the 20B model — the hosted API wins on raw dollars-per-token until your volume reaches fleet scale. A single 12GB card physically cannot push enough tokens per month to amortize itself against prices that low.
But that is the wrong comparison for most teams. The 20B model reasons at roughly o3-mini quality, and o3-mini's API costs $1.10 per million input tokens and $4.40 per million output tokens — about twenty times more. Against quality-equivalent frontier pricing, a cheap owned box breaks even at around 15 million tokens a month, a volume a single busy side project can clear. Self-hosting wins decisively there, and it bundles three things no per-token price can offer: data that never leaves your machines, a fixed monthly cost, and a license that lets you fine-tune and redistribute without asking anyone.
The 120B model is the mirror image: near-o4-mini quality, but it needs an 80GB-class card, so the hardware buy-in starts around four figures a month and the breakeven only arrives past half a billion tokens monthly. Read on for the full worked costing.
The VRAM math: why 20B fits in 12GB and 120B needs 80GB
OpenAI released gpt-oss-120b and gpt-oss-20b in August 2025 under the permissive Apache 2.0 license — its first open-weight release since GPT-2 in 2019. Both models were trained in MXFP4, a 4-bit microscaling floating-point format, which means the 4-bit weights are the native artifact, not a degraded post-training quantization. That single design decision is what makes the VRAM math work.
At 4 bits per parameter, each billion parameters needs roughly half a gigabyte for weights:
| Model | Parameters | Weights (MXFP4) | KV cache + overhead | Practical VRAM | Fits on |
|---|---|---|---|---|---|
| gpt-oss-20b | 21B | ~10.5 GB | ~2–5 GB | ~12–16 GB | RTX 3060 12GB (tight), any 16GB+ card |
| gpt-oss-120b | 117B | ~60 GB | ~5–11 GB | ~65–71 GB | Single 80GB card (H100, H200) or 96GB workstation card |
The 12GB fit is genuinely tight: community benchmarks confirm gpt-oss-20b runs on an RTX 3060 12GB at roughly 53–85 tokens per second via GGUF quants, but with a reduced context window once the KV cache starts competing with weights for room. A 16GB card (RTX 4060 Ti 16GB, or a Mac with unified memory) is the comfortable floor — OpenAI's own guidance names 16GB as the requirement. The 120B model at ~65GB needs a datacenter card or a 96GB workstation GPU; there is no consumer-card story for it.
Two quantization notes before the costing. First, because MXFP4 is native, the quality gap between the reference weights and the GGUF community quants (Unsloth publishes the popular ones) is smaller than the usual full-precision-to-4-bit drop — you are quantizing from an already-4-bit model, mostly trading context length and throughput, not reasoning ability. Second, longer contexts eat VRAM through the KV cache roughly linearly, so "fits in 12GB" always carries an asterisk for the context length you actually serve. Budget for your longest realistic conversation, not the 131K-token headline context window.
The serving path: from download to OpenAI-compatible endpoint
You have four credible serving stacks, in increasing order of operational seriousness:
Ollama is the one-liner: ollama run gpt-oss:20b pulls the weights and serves an OpenAI-compatible endpoint on localhost. It is the fastest way to verify the model answers your questions well enough to matter. It is not a multi-tenant production server.
llama.cpp with GGUF quants is the efficiency floor — the path behind those RTX 3060 community benchmarks. Maximum tokens per second per dollar of old hardware, at the cost of hand-rolled serving, batching, and API surface.
vLLM is the production answer, and OpenAI's own gpt-oss cookbook treats it as the reference path: vllm serve openai/gpt-oss-20b exposes a PagedAttention-backed, continuously-batched, OpenAI-compatible API. Continuous batching matters enormously for the economics below — instead of serving one request at a time, vLLM packs concurrent requests into the same GPU execution, so aggregate throughput across many small tenant requests lands several multiples above single-stream tokens-per-second. SGLang occupies the same tier with a different performance/feature tradeoff.
One gotcha applies to every stack: gpt-oss models were trained on the harmony response format for conversation structure, reasoning traces, and tool calls. If you call model.generate directly without the chat template, the model misbehaves; through Ollama, vLLM, SGLang, or Transformers' chat template, formatting is handled for you. Relatedly, both models expose low/medium/high reasoning-effort levels — a knob that trades output tokens (hence cost and latency) against answer depth, and one you should set deliberately per workload rather than leaving at default.
The worked costing: three volumes, both sizes, both sides
Assumptions, stated plainly so you can re-run the math: a 75/25 input/output token mix (typical for chat and agent workloads, which are context-heavy), 30-day months, and October 2026 prices.
Hosted per-token prices (verified): serverless gpt-oss-20b at ~$0.05/M input and ~$0.20/M tokens output blends to ~$0.09/M tokens; gpt-oss-120b at $0.15/M in and $0.60/M out (Together, Fireworks, Groq) blends to ~$0.26/M tokens. OpenRouter's September 2026 data shows 120b spanning $0.03–0.35 in and $0.17–0.95 out across providers, with customers actually paying a weighted average of ~$0.10 in / ~$0.39 out — so treat the headline numbers as the middle of a wide market, not a fixed tariff. Quality-equivalent frontier pricing: o3-mini and o4-mini both at $1.10/M in and $4.40/M out, blending to ~$1.93/M tokens.
Self-host fixed costs: a used RTX 3060 12GB at ~$300 amortized over 24 months plus ~170W of power at residential rates lands near $30/month all-in. A rented 24GB-class GPU (RTX 4090) runs roughly $0.44–0.89/hour depending on commitment — call it ~$365/month at a mid-range reserved rate for 24/7 serving. A 96GB Hetzner-class dedicated GPU box (GEX63 with an RTX PRO 6000 Blackwell) lists at €999/month, roughly $1,080.
| Monthly volume | Hosted 20b API | Owned 3060 20b (~$30/mo) | Hosted 120b API | 96GB box 120b (~$1,080/mo) |
|---|---|---|---|---|
| Side project · 50M tokens | ~$4 | $30 ( API wins 7x) | ~$13 | $1,080 (API wins 80x) |
| Growing SaaS · 500M tokens | ~$44 | $30 (self-host wins, but one card caps near ~155M tokens — you need 3–4 cards or one bigger GPU) | ~$131 | $1,080 (API wins 8x) |
| PaaS shared pool · 2B tokens | ~$175 | self-host wins: one 24GB card + vLLM batching (~$365/mo rented, less owned) | ~$525 | $1,080 (self-host wins past ~4B tokens at these API prices) |
Read the table carefully, because it contains the post's most important number: a single RTX 3060 at ~60 tokens/second can physically emit only about 155 million tokens per month running flat out. That caps its API-equivalent value near $14/month — below its own $30/month cost. Against like-for-like hosted weights, a small dedicated card never breaks even on token price alone. The breakeven only exists where throughput per dollar is higher (a 24GB card with vLLM continuous batching serving many tenants concurrently) or where the comparison is quality-equivalent frontier pricing: at ~$1.93/M tokens blended for o3-mini-class API, the $30 box breaks even near 15 million tokens a month, and the $1,080 120b box breaks even near 560 million tokens against o4-mini-class pricing.
Sensitivity check, because the mix assumption does real work: sliding from 75/25 to 50/50 input/output roughly doubles every hosted total (output tokens cost ~4x input), which pulls every breakeven proportionally closer. Output-heavy workloads — long reasoning traces at high effort, document generation — favor self-hosting sooner. Cache-heavy workloads with high prompt reuse favor the API side, where cached-input discounts (often 50%) apply and your fixed-cost box sits idle between bursts.
Why a PaaS would bundle the 20B anyway
If the like-for-like token math favors the API below fleet scale, why is the 20B model still the first credible "default tenant model" a self-hosted platform can bundle rather than proxy? Because token price was never the whole bid:
Apache 2.0 changes the product shape. No per-seat terms, no API ToS governing what tenants may build, no provider able to deprecate the model out from under you. You can fine-tune it on your domain, distill it, and ship the weights to customer premises — the license permits all of it, which is what makes a bundled default model a product decision instead of a reseller agreement.
Data never leaves the fleet. Every token sent to an inference API is data disclosed to a third party under their retention policy. For EU tenants under GDPR, for healthcare and finance workloads, and for any customer whose prompts contain their own customers' data, a model served on owned hardware in your datacenter is a compliance story no per-token discount can match. Hetzner makes the same point about its ISO 27001-certified EU datacenters for a reason.
Zero marginal cost per tenant. Once the pool exists, onboarding one more tenant to the shared model costs nothing at the margin until the pool saturates — versus a meter that starts running on every new signup. That flips the unit economics of free tiers and low-volume tenants: the API bill for a thousand idle-then-bursty tenants is unpredictable, while the shared pool is a flat line you provision once.
Offline and air-gap. A weights file on disk serves inference with no network path to any vendor — relevant for sovereign deployments, on-prem enterprise editions, and demos that must survive conference Wi-Fi.
The bundling thesis, stated honestly: you bundle the 20B because the license and the data story let you, and because at fleet-aggregate volume the fixed cost amortizes across tenants the API meter would charge individually — not because a single card beats $0.05-per-million tokens.
When to just pay the API
Intellectual honesty requires the other column. Pay per token when your volume is low or bursty — the side-project row of the table is not close, and idle GPUs earn nothing while the metered API bills zero at rest. Pay per token for the 120B model unless you already operate 80GB-class capacity; a four-figure monthly box for occasional Frontier-quality answers is the worst of both worlds. And pay per token when you have no GPU operations capacity: driver versions, MXFP4 kernel support, KV-cache sizing, and vLLM upgrades are a real on-call surface, and a team without it will spend more in engineering time than the API premium costs.
A reasonable ladder: prototype against the hosted API, move steady-state 20B volume to one owned card past ~15M quality-equivalent tokens a month, and graduate to a batched vLLM pool shared across tenants once aggregate demand justifies a 24GB+ card running hot. Each rung is a reversible decision — the weights and the OpenAI-compatible API surface are the same at every step, so migration is a URL change, not a rewrite.
Bex.co is the open-source, AI-native Render alternative — push a git repo, get a running HTTPS service on machines you own. Star the repo on GitHub or deploy your first app today.
