NVIDIA · Ada Lovelace · 2022
NVIDIA RTX 6000 Ada Generation for local AI
48GB in a 300W blower card that stacks well, though newer options undercut it.
Updated · Speeds marked ~ are estimates; seemethodology
- VRAM
- 48GB
- Bandwidth
- 960 GB/s
- Power
- 300W
- Runs well
- 8 models
RTX 6000 Ada
Typically about $4,750 used
Can the RTX 6000 Ada run LLMs?
The RTX 6000 Ada has 48GB of GDDR6 ECC and 960 GB/s of memory bandwidth, which decides how big a model fits and how fast it generates text. It comfortably runs models up to about 35B parameters: the largest in our catalog is Qwen3.6 35B-A3B at Q8_0 (41GB with 8K context, ~100 tokens/sec). Our top all-round pick for it is Qwen3.8 27B (Q8_0, ~25 tokens/sec). It can even run Llama 3.3 70B Instruct at Q3_K_M (~22 tokens/sec).
Recommendations
Best models for the RTX 6000 Ada
Best overall · also coding, reasoning, vision
Qwen3.8 27BQ8_0 · 32GB · ~25 tokens/sec
Runs wellRunner-up · also largest that fits
Qwen3.6 35B-A3BQ8_0 · 41GB · ~100 tokens/sec
Runs wellFastest good model
gpt-oss-20bMXFP4 · 15GB · ~130 tokens/sec
Runs well
Compatibility
What runs on the RTX 6000 Ada
Every current model in our catalog, at the best quantization that fits in 48GB with 8K tokens of context. "Partial offload" means part of the model runs from system RAM (we assume 64GB), which is much slower.
| Model | Parameters | Best quant | Memory | Speed | Verdict |
|---|---|---|---|---|---|
| Gemma 4 E4B | 8B | Q8_0 | 10 GB | ~75 tok/s | Runs well |
| Qwen3.5 9B | 9B | Q8_0 | 12 GB | ~66 tok/s | Runs well |
| Phi-4 | 14.7B | Q8_0 | 18 GB | ~47 tok/s | Runs well |
| gpt-oss-20b | 20.9B (3.6B active) | MXFP4 | 15 GB | ~130 tok/s | Runs well |
| Gemma 4 26B A4B | 26B (3.8B active) | Q8_0 | 30 GB | ~89 tok/s | Runs well |
| Qwen3.8 27B | 27B | Q8_0 | 32 GB | ~25 tok/s | Runs well |
| Gemma 4 31B | 31B | Q8_0 | 36 GB | ~23 tok/s | Runs well |
| Qwen3.6 35B-A3B | 35B (3B active) | Q8_0 | 41 GB | ~100 tok/s | Runs well |
| GLM-4.5-Air | 106B (12B active) | Q4_K_M | 68 GB | ~15 tok/s | Partial offload |
| Llama 4 Scout | 109B (17B active) | Q4_K_M | 70 GB | ~11 tok/s | Partial offload |
| gpt-oss-120b | 117B (5.1B active) | MXFP4 | 68 GB | ~37 tok/s | Partial offload |
| Qwen3.5 122B-A10B | 122B (10B active) | Q4_K_M | 77 GB | ~16 tok/s | Partial offload |
| DeepSeek V4 Flash | 284B (13B active) | — | — | — | Won't fitWhat it needs |
| GLM-5.3-Flash | 320B (18B active) | — | — | — | Won't fitWhat it needs |
| GLM-5.3 | 744B (40B active) | — | — | — | Won't fitWhat it needs |
Performance
RTX 6000 Ada vs. alternatives on Qwen3.8 27B
Generation speed in tokens per second for a single user. Token generation is limited mostly by memory bandwidth, which is why bandwidth predicts these results so well.
RTX 4090
Q4_K_M · 24GB
~47tok/s
RTX PRO 6000
Q8_0 · 96GB
~46tok/s
RTX 6000 Ada
Q8_0 · 48GB
~25tok/s
RTX A6000
Q8_0 · 48GB
~18tok/s
Measured
RTX 6000 Ada benchmark results
| Model | Quant | Runtime | Generation | Prompt | Source |
|---|---|---|---|---|---|
| Llama 2 7B Chat | Q4_0 | llama.cpp (CUDA) | 175 tok/s | 9230 tok/s | llama.cpp discussion #15013: Performance of llama.cpp on NVIDIA CUDA · October 2026 |
| Llama 3.1 8B Instruct | Q4_K_M | llama.cpp (CUDA) | 135 tok/s | 5790 tok/s | XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024 |
| Llama 3.1 8B Instruct | F16 | llama.cpp (CUDA) | 52 tok/s | 6665 tok/s | XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024 |
| Llama 3.3 70B Instruct | Q4_K_M | llama.cpp (CUDA) | 19 tok/s | 565 tok/s | XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024 |
| 4× · Llama 3.3 70B Instruct | Q4_K_M | llama.cpp (CUDA) | 18 tok/s | 655 tok/s | XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024 |
Our take
Should you buy the RTX 6000 Ada for AI?
The RTX 6000 Ada Generation carries 48GB of ECC GDDR6 on a 384-bit bus for 960 GB/s in a dual-slot blower card rated at 300W. Two cards hold 70B models at 8 bit, and four fit in a standard workstation without special cooling. It launched at $6,800, and used cards sold for about $4,750 to $7,400 in October 2026. At those prices, compare it with the 72GB RTX PRO 5000 Blackwell or a pair of 32GB cards.
What we like
- 48GB per card with ECC
- 300W blower design stacks densely
- 960 GB/s bandwidth
Watch out for
- Used prices remain high
- Slower than Blackwell workstation cards
- No NVLink
Best for: Quiet multi-GPU workstations · 70B models across two cards
Specifications
RTX 6000 Ada specs that matter for AI
- Memory
- 48GB GDDR6 ECC
- Memory bandwidth
- 960 GB/s
- Memory bus
- 384-bit
- Architecture
- Ada Lovelace
- Power
- 300W
- Interface
- PCIe 4.0 x16
- Released
- 2022
- Launch price
- $6,800
- Street price (used)
- ~$4,750 (2026-10)
- Software
- CUDA, Vulkan
Builds
PC builds with the RTX 6000 Ada
Cross-shop
Alternatives to the RTX 6000 Ada
Frequently asked questions
How much VRAM does the RTX 6000 Ada have?
The NVIDIA RTX 6000 Ada Generation has 48GB of GDDR6 ECC with 960 GB/s of memory bandwidth on a 384-bit bus.
What is the largest LLM the RTX 6000 Ada can run?
Fully in GPU memory, the largest model in our catalog that runs comfortably is Qwen3.6 35B-A3B (35B (3B active)) at Q8_0, using about 41GB with 8K context at ~100 tokens/sec.
Can the RTX 6000 Ada run Llama 3.3 70B?
Yes. At Q3_K_M it needs about 39GB with 8K context and runs at ~22 tokens/sec.
What is the best LLM to run on the RTX 6000 Ada?
Our top pick is Qwen3.8 27B at Q8_0, which fits in 48GB with room for 8K of context and runs at ~25 tokens/sec. See our full list of the best models for the RTX 6000 Ada for coding, reasoning, and vision picks.
Is the RTX 6000 Ada good for local AI in 2026?
48GB in a 300W blower card that stacks well, though newer options undercut it.