Hardware guide · October 2026
What GPU do you need to run Qwen3 235B-A22B (2507)?
Qwen3 235B-A22B (2507) is a 235B (22B active) mixture-of-experts model. Here is exactly how much memory it needs and which GPUs and Macs run it well.
The short answer
To run Qwen3 235B-A22B (2507) at Q4_K_M with 8K tokens of context you need about 149GB of VRAM (142.5GB of weights plus KV cache and overhead). No single consumer GPU can hold it. The most affordable setup is 8× RTX 3090 (~18 tokens/sec). On a Mac or other unified-memory system, the Ryzen AI Max+ PRO 495 runs it at ~11 tokens/sec.
- VRAM at Q4_K_M
- ~149GB
- GPUs that run it
- 4
- Parameters
- 235B
- CPU only (est.)
- ~3.3 tok/s
Our recommendations
The best GPUs for Qwen3 235B-A22B (2507)
Multi-GPU
8× RTX 3090192GB total · ~$8,400 used
- Quant
- Q4_K_M
- Speed (est.)
- ~18 tok/s
Split across cards with llama.cpp, vLLM, or ExLlama. Needs a board and PSU that can feed every card.
Faster multi-GPU
8× RTX 4090192GB total · ~$20,000 used
- Quant
- Q4_K_M
- Speed (est.)
- ~20 tok/s
Split across cards with llama.cpp, vLLM, or ExLlama. Needs a board and PSU that can feed every card.
Mac / unified memory
Ryzen AI Max+ PRO 495192GB unified · ~$6,800
- Quant
- Q3_K_M
- Speed (est.)
- ~11 tok/s
Lots of memory per dollar and silent, but slower than a GPU with the same capacity.
We may earn a commission when you buy through links on this page. It never changes what we recommend. How we make money
Memory
Qwen3 235B-A22B (2507) VRAM requirements
Memory needed at each quantization level. Q4_K_M is the best balance of size and quality for most people; move up to Q5 or Q6 if you have the room.
| Quantization | Quality | Weights | Total @ 8K ctx | Total @ 32K ctx |
|---|---|---|---|---|
| Q2_K | Rough | 88 GB | 93 GB | 98 GB |
| Q3_K_M | Usable | 115 GB | 120 GB | 125 GB |
| Q4_K_M | RecommendedDefault | 142 GB | 149 GB | 154 GB |
| Q5_K_M | Very good | 167 GB | 175 GB | 179 GB |
| Q6_K | Near-lossless | 194 GB | 202 GB | 207 GB |
| Q8_0 | Practically lossless | 250 GB | 259 GB | 264 GB |
| BF16 | Full precision | 470 GB | 486 GB | 491 GB |
Context
How context length changes the memory you need
Every token in the context window is stored in the KV cache. At Q4_K_M, with a 16-bit KV cache, Qwen3 235B-A22B (2507) needs:
| Context | KV cache | Total memory | With 8-bit KV |
|---|---|---|---|
| 4K tokens | 0.8 GB | 148.1 GB | 147.7 GB |
| 8K tokens | 1.6 GB | 148.9 GB | 148.1 GB |
| 16K tokens | 3.2 GB | 150.5 GB | 148.9 GB |
| 32K tokens | 6.3 GB | 153.7 GB | 150.5 GB |
| 64K tokens | 12.6 GB | 160.0 GB | 153.7 GB |
| 128K tokens | 25.2 GB | 172.6 GB | 160.0 GB |
| 256K tokens | 50.5 GB | 197.8 GB | 172.6 GB |
2 x 94 layers x 4 KV heads x 128 dims x 2 bytes = 192,512 bytes (188 KiB) per token: about 1.6GB at 8K and 50GB at the full 262K context.
Compatibility
Every GPU that runs Qwen3 235B-A22B (2507)
Each GPU at the highest-quality quantization that fits with 8K context, fastest first. Speeds are single-user estimates.
| GPU | Memory | Quant | Speed | Fit | Price |
|---|---|---|---|---|---|
| M5 Ultra 256GB | 256GB unified | Q4_K_M | ~27 tok/s | Runs well | ~$9,499 |
| M5 Ultra 512GB | 512GB unified | Q8_0 | ~16 tok/s | Runs well | — |
| M3 Ultra 512GB | 512GB unified | Q8_0 | ~11 tok/s | Runs well | $9,499 |
| Ryzen AI Max+ PRO 495 | 192GB unified | Q3_K_M | ~11 tok/s | Runs well | ~$6,800 |
Tips
Getting the most out of Qwen3 235B-A22B (2507)
- Start at Q4_K_M. It keeps nearly all of the quality at about a third of the 16-bit size. Step up to Q5_K_M or Q6_K only if it still fits with the context you need.
- Budget memory for context. The KV cache grows with every token. Quantizing it to 8-bit roughly halves that cost with little quality loss.
- Offload experts, not layers. For mixture-of-experts models, llama.cpp's
--n-cpu-moe(or-ottensor overrides) keeps attention on the GPU and moves expert weights to system RAM. That keeps speed usable when the model does not fit in VRAM. - Pick the right runtime. Ollama and LM Studio are easiest. llama.cpp gives the most control. vLLM or SGLang are best when several people or agents share one machine.
Want to try different quantizations, context lengths, or GPU counts? Use our VRAM calculator, or read the Qwen3 235B-A22B (2507) overview for setup commands and benchmarks.
Frequently asked questions
How much VRAM do I need to run Qwen3 235B-A22B (2507)?
About 149GB at Q4_K_M with an 8K context window. At Q8_0 (near-lossless) it needs about 259GB, and at Q3_K_M about 120GB with some quality loss. Longer contexts add 0.20GB per 1K tokens with a 16-bit KV cache.
Can I run Qwen3 235B-A22B (2507) on a 24GB GPU?
No. Qwen3 235B-A22B (2507) needs about 149GB at Q4_K_M, which does not fit on a 24GB card like the RTX 3090 or RTX 4090, even with CPU offload on a 64GB system.
Can I run Qwen3 235B-A22B (2507) on a Mac?
Yes. With 192GB of unified memory or more (for example the Ryzen AI Max+ PRO 495), macOS can give the GPU enough memory to run it at Q3_K_M, at about ~11 tokens/sec. MLX builds are usually fastest on Apple silicon.
Can I run Qwen3 235B-A22B (2507) on CPU only, without a GPU?
Technically yes on a server with enough RAM, but at about 149GB it needs 128GB+ of RAM and will be slow on a desktop.