Hardware guide · October 2026
What GPU do you need to run gpt-oss-20b?
gpt-oss-20b is a 20.9B (3.6B active) mixture-of-experts model. Here is exactly how much memory it needs and which GPUs and Macs run it well.
The short answer
To run gpt-oss-20b at MXFP4 with 8K tokens of context you need about 15GB of VRAM (13.8GB of weights plus KV cache and overhead). The cheapest way in is a 16GB card like the RTX 4060 Ti 16GB (~$380 used). For the best speed per dollar, get the RTX 3090 (~120 tokens/sec). On a Mac or other unified-memory system, the Ryzen AI Max+ 395 runs it at ~41 tokens/sec.
- VRAM at MXFP4
- ~15GB
- GPUs that run it
- 31
- Parameters
- 20.9B
- CPU only (est.)
- ~18 tok/s
Our recommendations
The best GPUs for gpt-oss-20b
Minimum
RTX 4060 Ti 16GB16GB · ~$380 used
- Quant
- MXFP4
- Speed (est.)
- ~51 tok/s
The smallest card that runs it at MXFP4 with 8K context.
Best value
RTX 309024GB · ~$1,050 used
- Quant
- MXFP4
- Speed (est.)
- ~120 tok/s
The most tokens per second per dollar among cards with room to spare.
Fastest
RTX 509032GB · ~$4,300
- Quant
- MXFP4
- Speed (est.)
- ~190 tok/s
The fastest single GPU in our database for this model.
Mac / unified memory
Ryzen AI Max+ 395128GB unified · ~$3,300
- Quant
- MXFP4
- Speed (est.)
- ~41 tok/s
Lots of memory per dollar and silent, but slower than a GPU with the same capacity.
We may earn a commission when you buy through links on this page. It never changes what we recommend. How we make money
Memory
gpt-oss-20b VRAM requirements
gpt-oss-20b is released in MXFP4, so its memory use depends mostly on context length.
| Quantization | Quality | Weights | Total @ 8K ctx | Total @ 32K ctx |
|---|---|---|---|---|
| MXFP4 | NativeDefault | 14 GB | 15 GB | 16 GB |
Context
How context length changes the memory you need
Every token in the context window is stored in the KV cache. At MXFP4, with a 16-bit KV cache, gpt-oss-20b needs:
| Context | KV cache | Total memory | With 8-bit KV |
|---|---|---|---|
| 4K tokens | 0.1 GB | 14.9 GB | 14.9 GB |
| 8K tokens | 0.2 GB | 15.0 GB | 14.9 GB |
| 16K tokens | 0.4 GB | 15.2 GB | 15.0 GB |
| 32K tokens | 0.8 GB | 15.6 GB | 15.2 GB |
| 64K tokens | 1.6 GB | 16.4 GB | 15.6 GB |
| 128K tokens | 3.2 GB | 18.0 GB | 16.4 GB |
24 layers alternate between a 128-token sliding window and full attention, so only the 12 full-attention layers grow with context: 2 x 12 x 8 KV heads x 64 dims x 2 bytes = 24,576 bytes (24 KiB) per token, about 3.2GB at the full 128K. The sliding layers add a negligible fixed buffer.
Compatibility
Every GPU that runs gpt-oss-20b
Each GPU at the highest-quality quantization that fits with 8K context, fastest first. Speeds are single-user estimates.
| GPU | Memory | Quant | Speed | Fit | Price |
|---|---|---|---|---|---|
| RTX PRO 6000 | 96GB | MXFP4 | ~190 tok/s | Runs well | ~$16,000 |
| RTX 5090 | 32GB | MXFP4 | ~190 tok/s | Runs well | ~$4,300 |
| RTX PRO 5000 72GB | 72GB | MXFP4 | ~160 tok/s | Runs well | ~$9,100 |
| RTX 4090 | 24GB | MXFP4 | ~135 tok/s | Runs well | ~$2,500 used |
| RTX 6000 Ada | 48GB | MXFP4 | ~130 tok/s | Runs well | ~$4,750 used |
| RTX 3090 | 24GB | MXFP4 | ~120 tok/s | Runs well | ~$1,050 used |
| M5 Ultra 512GB | 512GB unified | MXFP4 | ~105 tok/s | Runs well | — |
| M5 Ultra 256GB | 256GB unified | MXFP4 | ~105 tok/s | Runs well | ~$9,499 |
| RTX A6000 | 48GB | MXFP4 | ~105 tok/s | Runs well | ~$4,400 used |
| V100 32GB | 32GB | MXFP4 | ~100 tok/s | Runs well | ~$700 used |
| RX 7900 XTX | 24GB | MXFP4 | ~99 tok/s | Runs well | ~$1,050 used |
| M3 Ultra 512GB | 512GB unified | MXFP4 | ~80 tok/s | Runs well | $9,499 |
| R9700 | 32GB | MXFP4 | ~75 tok/s | Runs well | ~$1,800 |
| MI50 32GB | 32GB | MXFP4 | ~74 tok/s | Runs well | ~$600 used |
| M5 Max 128GB | 128GB unified | MXFP4 | ~73 tok/s | Runs well | ~$5,100 |
| M4 Max 128GB | 128GB unified | MXFP4 | ~67 tok/s | Runs well | $3,499 |
| RTX 4000 Ada | 20GB | MXFP4 | ~62 tok/s | Runs well | $1,250 |
| Arc Pro B70 | 32GB | MXFP4 | ~59 tok/s | Runs well | ~$1,300 |
| Arc Pro B60 | 24GB | MXFP4 | ~48 tok/s | Runs well | ~$650 |
| Ryzen AI Max+ PRO 495 | 192GB unified | MXFP4 | ~43 tok/s | Runs well | ~$6,800 |
| DGX Spark | 128GB unified | MXFP4 | ~43 tok/s | Runs well | ~$6,950 |
| Ryzen AI Max+ 395 | 128GB unified | MXFP4 | ~41 tok/s | Runs well | ~$3,300 |
| Tesla P40 | 24GB | MXFP4 | ~41 tok/s | Runs well | ~$230 used |
| RTX 5080 | 16GB | MXFP4 | ~130 tok/s | Fits, tight | ~$1,600 |
| RTX 5070 Ti | 16GB | MXFP4 | ~125 tok/s | Fits, tight | ~$1,200 |
| RTX 4080 Super | 16GB | MXFP4 | ~110 tok/s | Fits, tight | ~$1,150 used |
| RTX 4070 Ti Super | 16GB | MXFP4 | ~100 tok/s | Fits, tight | ~$900 used |
| RX 9070 XT | 16GB | MXFP4 | ~75 tok/s | Fits, tight | ~$800 |
| RTX 5060 Ti 16GB | 16GB | MXFP4 | ~74 tok/s | Fits, tight | ~$620 |
| RTX 4060 Ti 16GB | 16GB | MXFP4 | ~51 tok/s | Fits, tight | ~$380 used |
| RX 9060 XT 16GB | 16GB | MXFP4 | ~44 tok/s | Fits, tight | ~$470 |
With CPU offload (partial offload, assuming 64GB of system RAM): RTX 5070 (~47 tok/s), RTX 4070 (~43 tok/s), RTX 3060 12GB (~37 tok/s), Arc B580 (~33 tok/s), RTX 5060 (~28 tok/s).
Tips
Getting the most out of gpt-oss-20b
- Budget memory for context. The KV cache grows with every token. Quantizing it to 8-bit roughly halves that cost with little quality loss.
- Offload experts, not layers. For mixture-of-experts models, llama.cpp's
--n-cpu-moe(or-ottensor overrides) keeps attention on the GPU and moves expert weights to system RAM. That keeps speed usable when the model does not fit in VRAM. - Pick the right runtime. Ollama and LM Studio are easiest. llama.cpp gives the most control. vLLM or SGLang are best when several people or agents share one machine.
Want to try different quantizations, context lengths, or GPU counts? Use our VRAM calculator, or read the gpt-oss-20b overview for setup commands and benchmarks.
Frequently asked questions
How much VRAM do I need to run gpt-oss-20b?
About 15GB at MXFP4 with an 8K context window. gpt-oss-20b ships in MXFP4, so there are no smaller official quantizations. Longer contexts add 0.03GB per 1K tokens with a 16-bit KV cache.
Can I run gpt-oss-20b on a 12GB GPU?
Only with partial CPU offload. On a 12GB card like the RTX 3060 12GB, about 25% of the weights would sit in system RAM and generation drops to about 47 tokens/sec with CPU offload.
Can I run gpt-oss-20b on a Mac?
Yes. With 128GB of unified memory or more (for example the Ryzen AI Max+ 395), macOS can give the GPU enough memory to run it at MXFP4, at about ~41 tokens/sec. MLX builds are usually fastest on Apple silicon.
Can I run gpt-oss-20b on CPU only, without a GPU?
Yes, if you have at least 32GB of system RAM. On a desktop with dual-channel DDR5, expect roughly 18 tokens/sec, which is better than you might expect because only the active experts are read for each token. A GPU is recommended for interactive use.