NVIDIA · Blackwell · 2025
NVIDIA RTX PRO 6000 Blackwell for local AI
The most memory on a single PCIe card at RTX 5090-class bandwidth, at a very steep price.
Updated · Speeds marked ~ are estimates; seemethodology
- VRAM
- 96GB
- Bandwidth
- 1,792 GB/s
- Power
- 600W
- Runs well
- 12 models
RTX PRO 6000
Typically about $16,000
Can the RTX PRO 6000 run LLMs?
The RTX PRO 6000 has 96GB of GDDR7 ECC and 1,792 GB/s of memory bandwidth, which decides how big a model fits and how fast it generates text. It comfortably runs models up to about 122B parameters: the largest in our catalog is Qwen3.5 122B-A10B at Q4_K_M (77GB with 8K context, ~105 tokens/sec). Our top all-round pick for it is Qwen3.8 27B (Q8_0, ~46 tokens/sec).
Recommendations
Best models for the RTX PRO 6000
Best overall · also coding, reasoning, vision
Qwen3.8 27BQ8_0 · 32GB · ~46 tokens/sec
Runs wellRunner-up · also largest that fits
Qwen3.5 122B-A10BQ4_K_M · 77GB · ~105 tokens/sec
Runs wellFastest good model
gpt-oss-20bMXFP4 · 15GB · ~190 tokens/sec
Runs well
Compatibility
What runs on the RTX PRO 6000
Every current model in our catalog, at the best quantization that fits in 96GB with 8K tokens of context. "Partial offload" means part of the model runs from system RAM (we assume 64GB), which is much slower.
| Model | Parameters | Best quant | Memory | Speed | Verdict |
|---|---|---|---|---|---|
| Gemma 4 E4B | 8B | Q8_0 | 10 GB | ~130 tok/s | Runs well |
| Qwen3.5 9B | 9B | Q8_0 | 12 GB | ~115 tok/s | Runs well |
| Phi-4 | 14.7B | Q8_0 | 18 GB | ~84 tok/s | Runs well |
| gpt-oss-20b | 20.9B (3.6B active) | MXFP4 | 15 GB | ~190 tok/s | Runs well |
| Gemma 4 26B A4B | 26B (3.8B active) | Q8_0 | 30 GB | ~140 tok/s | Runs well |
| Qwen3.8 27B | 27B | Q8_0 | 32 GB | ~46 tok/s | Runs well |
| Gemma 4 31B | 31B | Q8_0 | 36 GB | ~42 tok/s | Runs well |
| Qwen3.6 35B-A3B | 35B (3B active) | Q8_0 | 41 GB | ~155 tok/s | Runs well |
| GLM-4.5-Air | 106B (12B active) | Q5_K_M | 80 GB | ~80 tok/s | Runs well |
| Llama 4 Scout | 109B (17B active) | Q5_K_M | 81 GB | ~61 tok/s | Runs well |
| gpt-oss-120b | 117B (5.1B active) | MXFP4 | 68 GB | ~170 tok/s | Runs well |
| Qwen3.5 122B-A10B | 122B (10B active) | Q4_K_M | 77 GB | ~105 tok/s | Runs well |
| DeepSeek V4 Flash | 284B (13B active) | — | — | — | Won't fitWhat it needs |
| GLM-5.3-Flash | 320B (18B active) | — | — | — | Won't fitWhat it needs |
| GLM-5.3 | 744B (40B active) | — | — | — | Won't fitWhat it needs |
Performance
RTX PRO 6000 vs. alternatives on Qwen3.8 27B
Generation speed in tokens per second for a single user. Token generation is limited mostly by memory bandwidth, which is why bandwidth predicts these results so well.
RTX 5090
Q6_K · 32GB
~59tok/s
RTX PRO 6000
Q8_0 · 96GB
~46tok/s
RTX PRO 5000 72GB
Q8_0 · 72GB
~35tok/s
RTX 6000 Ada
Q8_0 · 48GB
~25tok/s
Measured
RTX PRO 6000 benchmark results
| Model | Quant | Runtime | Generation | Prompt | Source |
|---|---|---|---|---|---|
| Llama 2 7B Chat | Q4_0 | llama.cpp (CUDA) | 275 tok/s | 14855 tok/s | llama.cpp discussion #15013: Performance of llama.cpp on NVIDIA CUDA · October 2026 |
Our take
Should you buy the RTX PRO 6000 for AI?
The RTX PRO 6000 Blackwell puts 96GB of ECC GDDR7 on a 512-bit bus for 1,792 GB/s, the same bandwidth as the RTX 5090 with three times the memory. One card holds 70B dense models at 8 bit or 100B-class MoE models at 4 bit, and two or four cards reach the largest open models. NVIDIA sells a 600W Workstation Edition, a 300W Max-Q blower version for multi-GPU towers, and a Server Edition for passively cooled chassis. The memory shortage pushed listings to around $16,000 in October 2026, close to double its 2025 street price.
What we like
- 96GB on one card avoids multi-GPU complexity
- 1,792 GB/s matches the RTX 5090
- 300W Max-Q version fits four to a workstation
- ECC memory and professional drivers
Watch out for
- Around $16,000 in October 2026
- The 600W Workstation Edition needs serious cooling
Best for: Running 70B to 120B models on a single card · Multi-GPU workstations for the largest open models · Fine-tuning and serving in professional settings
Specifications
RTX PRO 6000 specs that matter for AI
- Memory
- 96GB GDDR7 ECC
- Memory bandwidth
- 1,792 GB/s
- Memory bus
- 512-bit
- Architecture
- Blackwell
- Power
- 600W
- Interface
- PCIe 5.0 x16
- Released
- 2025
- Street price (new)
- ~$16,000 (2026-10)
- Software
- CUDA, Vulkan
The 300W Max-Q Workstation Edition has the same 96GB and bandwidth.
Sources: NVIDIA RTX PRO 6000 Blackwell Server Edition product page, Wikipedia, Blackwell (microarchitecture)
Builds
PC builds with the RTX PRO 6000
Cross-shop
Alternatives to the RTX PRO 6000
Frequently asked questions
How much VRAM does the RTX PRO 6000 have?
The NVIDIA RTX PRO 6000 Blackwell has 96GB of GDDR7 ECC with 1,792 GB/s of memory bandwidth on a 512-bit bus.
What is the largest LLM the RTX PRO 6000 can run?
Fully in GPU memory, the largest model in our catalog that runs comfortably is Qwen3.5 122B-A10B (122B (10B active)) at Q4_K_M, using about 77GB with 8K context at ~105 tokens/sec.
Can the RTX PRO 6000 run Llama 3.3 70B?
Yes. At Q8_0 it needs about 81GB with 8K context and runs at ~19 tokens/sec.
What is the best LLM to run on the RTX PRO 6000?
Our top pick is Qwen3.8 27B at Q8_0, which fits in 96GB with room for 8K of context and runs at ~46 tokens/sec. See our full list of the best models for the RTX PRO 6000 for coding, reasoning, and vision picks.
Is the RTX PRO 6000 good for local AI in 2026?
The most memory on a single PCIe card at RTX 5090-class bandwidth, at a very steep price.