Hardware guide · October 2026
What GPU do you need to run Llama 4 Scout?
Llama 4 Scout is a 109B (17B active) mixture-of-experts model. Here is exactly how much memory it needs and which GPUs and Macs run it well.
The short answer
To run Llama 4 Scout at Q4_K_M with 8K tokens of context you need about 70GB of VRAM (65.4GB of weights plus KV cache and overhead). The cheapest way in is a 72GB card like the RTX PRO 5000 72GB (~$9,100). On a Mac or other unified-memory system, the Ryzen AI Max+ 395 runs it at ~9.3 tokens/sec.
- VRAM at Q4_K_M
- ~70GB
- GPUs that run it
- 10
- Parameters
- 109B
- CPU only (est.)
- ~4.3 tok/s
Our recommendations
The best GPUs for Llama 4 Scout
Minimum
RTX PRO 5000 72GB72GB · ~$9,100
- Quant
- Q3_K_M
- Speed (est.)
- ~66 tok/s
The smallest card that runs it at Q3_K_M with 8K context.
Multi-GPU
3× RTX 309072GB total · ~$3,150 used
- Quant
- Q4_K_M
- Speed (est.)
- ~30 tok/s
Split across cards with llama.cpp, vLLM, or ExLlama. Needs a board and PSU that can feed every card.
Faster multi-GPU
3× RTX 409072GB total · ~$7,500 used
- Quant
- Q4_K_M
- Speed (est.)
- ~34 tok/s
Split across cards with llama.cpp, vLLM, or ExLlama. Needs a board and PSU that can feed every card.
Mac / unified memory
Ryzen AI Max+ 395128GB unified · ~$3,300
- Quant
- Q5_K_M
- Speed (est.)
- ~9.3 tok/s
Lots of memory per dollar and silent, but slower than a GPU with the same capacity.
We may earn a commission when you buy through links on this page. It never changes what we recommend. How we make money
Memory
Llama 4 Scout VRAM requirements
Memory needed at each quantization level. Q4_K_M is the best balance of size and quality for most people; move up to Q5 or Q6 if you have the room.
| Quantization | Quality | Weights | Total @ 8K ctx | Total @ 32K ctx |
|---|---|---|---|---|
| Q2_K | Rough | 40 GB | 44 GB | 45 GB |
| Q3_K_M | Usable | 53 GB | 56 GB | 58 GB |
| Q4_K_M | RecommendedDefault | 65 GB | 70 GB | 71 GB |
| Q5_K_M | Very good | 77 GB | 81 GB | 83 GB |
| Q6_K | Near-lossless | 89 GB | 94 GB | 95 GB |
| Q8_0 | Practically lossless | 115 GB | 120 GB | 121 GB |
| BF16 | Full precision | 216 GB | 224 GB | 225 GB |
Context
How context length changes the memory you need
Every token in the context window is stored in the KV cache. At Q4_K_M, with a 16-bit KV cache, Llama 4 Scout needs:
| Context | KV cache | Total memory | With 8-bit KV |
|---|---|---|---|
| 4K tokens | 1.4 GB | 69.3 GB | 68.6 GB |
| 8K tokens | 1.6 GB | 69.5 GB | 68.7 GB |
| 16K tokens | 2.0 GB | 69.9 GB | 68.9 GB |
| 32K tokens | 2.8 GB | 70.7 GB | 69.3 GB |
| 64K tokens | 4.4 GB | 72.3 GB | 70.1 GB |
| 128K tokens | 7.6 GB | 75.6 GB | 71.7 GB |
| 256K tokens | 14.1 GB | 82.0 GB | 75.0 GB |
48 layers, but three of every four use chunked local attention over 8,192-token chunks, and only the 12 global (NoPE) layers keep the whole context: 2 x 12 x 8 KV heads x 128 dims x 2 bytes = 49,152 bytes per token. The chunked layers add a fixed buffer of about 1.2GB (36 x 8 x 128 x 2 x 2 bytes x 8,192 tokens). Runtimes without chunked-attention support cache all 48 layers, which is 196,608 bytes per token.
Compatibility
Every GPU that runs Llama 4 Scout
Each GPU at the highest-quality quantization that fits with 8K context, fastest first. Speeds are single-user estimates.
| GPU | Memory | Quant | Speed | Fit | Price |
|---|---|---|---|---|---|
| RTX PRO 5000 72GB | 72GB | Q3_K_M | ~66 tok/s | Runs well | ~$9,100 |
| RTX PRO 6000 | 96GB | Q5_K_M | ~61 tok/s | Runs well | ~$16,000 |
| M5 Ultra 512GB | 512GB unified | Q8_0 | ~21 tok/s | Runs well | — |
| M5 Ultra 256GB | 256GB unified | Q8_0 | ~21 tok/s | Runs well | ~$9,499 |
| M5 Max 128GB | 128GB unified | Q5_K_M | ~19 tok/s | Runs well | ~$5,100 |
| M4 Max 128GB | 128GB unified | Q5_K_M | ~17 tok/s | Runs well | $3,499 |
| M3 Ultra 512GB | 512GB unified | Q8_0 | ~14 tok/s | Runs well | $9,499 |
| Ryzen AI Max+ 395 | 128GB unified | Q5_K_M | ~9.3 tok/s | Runs well | ~$3,300 |
| DGX Spark | 128GB unified | Q6_K | ~8.6 tok/s | Runs well | ~$6,950 |
| Ryzen AI Max+ PRO 495 | 192GB unified | Q8_0 | ~6.7 tok/s | Runs well | ~$6,800 |
With CPU offload (partial offload, assuming 64GB of system RAM): RTX 6000 Ada (~11 tok/s), RTX A6000 (~10 tok/s), RTX 5090 (~7.2 tok/s), V100 32GB (~6.8 tok/s), MI50 32GB (~6.6 tok/s), R9700 (~6.6 tok/s), Arc Pro B70 (~6.3 tok/s), RTX 4090 (~5.9 tok/s) and 4 more.
Tips
Getting the most out of Llama 4 Scout
- Start at Q4_K_M. It keeps nearly all of the quality at about a third of the 16-bit size. Step up to Q5_K_M or Q6_K only if it still fits with the context you need.
- Budget memory for context. The KV cache grows with every token. Quantizing it to 8-bit roughly halves that cost with little quality loss.
- Offload experts, not layers. For mixture-of-experts models, llama.cpp's
--n-cpu-moe(or-ottensor overrides) keeps attention on the GPU and moves expert weights to system RAM. That keeps speed usable when the model does not fit in VRAM. - Pick the right runtime. Ollama and LM Studio are easiest. llama.cpp gives the most control. vLLM or SGLang are best when several people or agents share one machine.
Want to try different quantizations, context lengths, or GPU counts? Use our VRAM calculator, or read the Llama 4 Scout overview for setup commands and benchmarks.
Frequently asked questions
How much VRAM do I need to run Llama 4 Scout?
About 70GB at Q4_K_M with an 8K context window. At Q8_0 (near-lossless) it needs about 120GB, and at Q3_K_M about 56GB with some quality loss. Longer contexts add 0.20GB per 1K tokens with a 16-bit KV cache.
Can I run Llama 4 Scout on a 24GB GPU?
Only with partial CPU offload. On a 24GB card like the RTX 3090 or RTX 4090, about 70% of the weights would sit in system RAM and generation drops to about 5.9 tokens/sec with CPU offload.
Can I run Llama 4 Scout on a Mac?
Yes. With 128GB of unified memory or more (for example the Ryzen AI Max+ 395), macOS can give the GPU enough memory to run it at Q5_K_M, at about ~9.3 tokens/sec. MLX builds are usually fastest on Apple silicon.
Can I run Llama 4 Scout on CPU only, without a GPU?
Yes, if you have at least 80GB of system RAM. On a desktop with dual-channel DDR5, expect roughly 4.3 tokens/sec, which is better than you might expect because only the active experts are read for each token. A GPU is strongly recommended for interactive use.