GPU database

Every GPU for local AI, ranked by what it can run

VRAM decides which models fit. Bandwidth decides how fast they run. Here is how every card and unified-memory system stacks up.

Which GPU should you buy?

For most people, a used RTX 3090 is still the best value: 24GB runs 32B models at a usable speed. On a tighter budget, a 16GB card like the RTX 5060 Ti 16GB handles 12–14B models well. For the fastest single card, the RTX 5090 has 32GB and the most bandwidth of any consumer GPU. Read our full buying guide for picks at every budget.

Unified-memory systems

Macs, Strix Halo mini PCs, and DGX Spark share one big pool of memory between CPU and GPU. Slower than a big GPU, but they hold much larger models.

AMD2025

128GB unified

LPDDR5X-8000 · 256 GB/s

Ryzen AI Max+ 395

~$3,300

Apple2026

512GB unified

LPDDR5X · 1,228 GB/s

M5 Ultra 512GB

NVIDIA2025

128GB unified

LPDDR5X · 273 GB/s

DGX Spark

~$6,950

Apple2026

128GB unified

LPDDR5X · 614 GB/s

M5 Max 128GB

~$5,100

Apple2025

512GB unified

LPDDR5 · 819 GB/s

M3 Ultra 512GB

$9,499 MSRP

Apple2026

256GB unified

LPDDR5X · 1,228 GB/s

M5 Ultra 256GB

~$9,499

Apple2025

128GB unified

LPDDR5X · 546 GB/s

M4 Max 128GB

$3,499 MSRP

AMD2026

192GB unified

LPDDR5X-8533 · 273 GB/s

Ryzen AI Max+ PRO 495

~$6,800

48GB and up

Workstation and datacenter cards that run 70B-class models on a single GPU.

NVIDIA2025

96GB

GDDR7 ECC · 1,792 GB/s

RTX PRO 6000

~$16,000

NVIDIA2020

48GB

GDDR6 ECC · 768 GB/s

RTX A6000

~$4,400 used

NVIDIA2022

48GB

GDDR6 ECC · 960 GB/s

RTX 6000 Ada

~$4,750 used

NVIDIA2025

72GB

GDDR7 ECC · 1,344 GB/s

RTX PRO 5000 72GB

~$9,100

32GB

Room for 32B models at high quality with long context, or mid-size MoE models.

NVIDIA2025

32GB

GDDR7 · 1,792 GB/s

RTX 5090

~$4,300

AMD2018

32GB

HBM2 · 1,024 GB/s

MI50 32GB

~$600 used

AMD2025

32GB

GDDR6 · 640 GB/s

R9700

~$1,800

Intel2026

32GB

GDDR6 ECC · 608 GB/s

Arc Pro B70

~$1,300

NVIDIA2018

32GB

HBM2 · 900 GB/s

V100 32GB

~$700 used

20–24GB

The sweet spot for local AI: 27–32B models at Q4 with useful context.

NVIDIA2020

24GB

GDDR6X · 936 GB/s

RTX 3090

~$1,050 used

NVIDIA2022

24GB

GDDR6X · 1,008 GB/s

RTX 4090

~$2,500 used

AMD2022

24GB

GDDR6 · 960 GB/s

RX 7900 XTX

~$1,050 used

NVIDIA2016

24GB

GDDR5 · 346 GB/s

Tesla P40

~$230 used

Intel2025

24GB

GDDR6 · 456 GB/s

Arc Pro B60

~$650

NVIDIA2023

20GB

GDDR6 ECC · 360 GB/s

RTX 4000 Ada

$1,250 MSRP

16GB

Comfortable with 12–14B models and small MoE models; 24B fits at lower quants.

NVIDIA2025

16GB

GDDR7 · 448 GB/s

RTX 5060 Ti 16GB

~$620

NVIDIA2025

16GB

GDDR7 · 896 GB/s

RTX 5070 Ti

~$1,200

NVIDIA2023

16GB

GDDR6 · 288 GB/s

RTX 4060 Ti 16GB

~$380 used

NVIDIA2025

16GB

GDDR7 · 960 GB/s

RTX 5080

~$1,600

AMD2025

16GB

GDDR6 · 640 GB/s

RX 9070 XT

~$800

NVIDIA2024

16GB

GDDR6X · 672 GB/s

RTX 4070 Ti Super

~$900 used

AMD2025

16GB

GDDR6 · 320 GB/s

RX 9060 XT 16GB

~$470

NVIDIA2024

16GB

GDDR6X · 736 GB/s

RTX 4080 Super

~$1,150 used

12GB and under

Entry level. Best with 4–9B models; 12–14B fits at Q4 with short context.

NVIDIA2021

12GB

GDDR6 · 360 GB/s

RTX 3060 12GB

~$270 used

Intel2024

12GB

GDDR6 · 456 GB/s

Arc B580

~$330

NVIDIA2025

12GB

GDDR7 · 672 GB/s

RTX 5070

~$800

NVIDIA2023

12GB

GDDR6X · 504 GB/s

RTX 4070

~$560 used

NVIDIA2025

8GB

GDDR7 · 448 GB/s

RTX 5060

~$450

Compare

Full spec table

Click a column to sort. Speed is an estimate for Llama 3.1 8B Instruct at Q4_K_M, a good proxy for memory bandwidth in practice.

GPU	Memory	Bandwidth	8B speed	Largest model that runs well	Price	$/GB	Power
M5 Ultra 512GBApple	512GB*	1,228 GB/s	~115 tok/s	GLM-5.3-Flash Q6_K	—Configure at Apple ↗	—	480W
M3 Ultra 512GBApple	512GB*	819 GB/s	~80 tok/s	GLM-5.3-Flash Q6_K	$9,499 MSRPCheck eBay ↗	$19	480W
M5 Ultra 256GBApple	256GB*	1,228 GB/s	~115 tok/s	GLM-5.3-Flash Q3_K_M	~$9,499Configure at Apple ↗	$37	480W
Ryzen AI Max+ PRO 495AMD	192GB*	273 GB/s	~39 tok/s	Qwen3.5 122B-A10B Q6_K	~$6,800Check Amazon ↗	$35	120W
M5 Max 128GBApple	128GB*	614 GB/s	~72 tok/s	Qwen3.5 122B-A10B Q4_K_M	~$5,100Configure at Apple ↗	$40	480W
M4 Max 128GBApple	128GB*	546 GB/s	~65 tok/s	Qwen3.5 122B-A10B Q4_K_M	$3,499 MSRPCheck eBay ↗	$27	480W
DGX SparkNVIDIA	128GB*	273 GB/s	~39 tok/s	Qwen3.5 122B-A10B Q6_K	~$6,950View at NVIDIA ↗	$54	240W
Ryzen AI Max+ 395AMD	128GB*	256 GB/s	~37 tok/s	Qwen3.5 122B-A10B Q4_K_M	~$3,300Check Amazon ↗	$26	120W
RTX PRO 6000NVIDIA	96GB	1,792 GB/s	~225 tok/s	Qwen3.5 122B-A10B Q4_K_M	~$16,000Check Amazon ↗	$167	600W
RTX PRO 5000 72GBNVIDIA	72GB	1,344 GB/s	~180 tok/s	Qwen3.5 122B-A10B Q3_K_M	~$9,100Check Amazon ↗	$126	300W
RTX 6000 AdaNVIDIA	48GB	960 GB/s	~135 tok/s	Qwen3.6 35B-A3B Q8_0	~$4,750 usedCheck Amazon ↗	$99	300W
RTX A6000NVIDIA	48GB	768 GB/s	~100 tok/s	Qwen3.6 35B-A3B Q8_0	~$4,400 usedCheck Amazon ↗	$92	300W
RTX 5090NVIDIA	32GB	1,792 GB/s	~225 tok/s	Qwen3.6 35B-A3B Q5_K_M	~$4,300Check Amazon ↗	$134	575W
V100 32GBNVIDIA	32GB	900 GB/s	~105 tok/s	Qwen3.6 35B-A3B Q5_K_M	~$700 usedCheck Amazon ↗	$22	300W
MI50 32GBAMD	32GB	1,024 GB/s	~79 tok/s	Qwen3.6 35B-A3B Q5_K_M	~$600 usedCheck Amazon ↗	$19	300W
R9700AMD	32GB	640 GB/s	~75 tok/s	Qwen3.6 35B-A3B Q5_K_M	~$1,800Check Amazon ↗	$56	300W
Arc Pro B70Intel	32GB	608 GB/s	~60 tok/s	Qwen3.6 35B-A3B Q5_K_M	~$1,300Check Amazon ↗	$41	230W
RTX 4090NVIDIA	24GB	1,008 GB/s	~140 tok/s	Qwen3.6 35B-A3B Q3_K_M	~$2,500 usedCheck Amazon ↗	$104	450W
RTX 3090NVIDIA	24GB	936 GB/s	~120 tok/s	Qwen3.6 35B-A3B Q3_K_M	~$1,050 usedCheck Amazon ↗	$44	350W
RX 7900 XTXAMD	24GB	960 GB/s	~105 tok/s	Qwen3.6 35B-A3B Q3_K_M	~$1,050 usedCheck Amazon ↗	$44	355W
Arc Pro B60Intel	24GB	456 GB/s	~46 tok/s	Qwen3.6 35B-A3B Q3_K_M	~$650Check Amazon ↗	$27	200W
Tesla P40NVIDIA	24GB	346 GB/s	~37 tok/s	Qwen3.6 35B-A3B Q3_K_M	~$230 usedCheck Amazon ↗	$10	250W
RTX 4000 AdaNVIDIA	20GB	360 GB/s	~55 tok/s	Gemma 4 26B A4B Q4_K_M	$1,250 MSRPCheck Amazon ↗	$63	130W
RTX 5080NVIDIA	16GB	960 GB/s	~135 tok/s	Phi-4 Q5_K_M	~$1,600Check Amazon ↗	$100	360W
RTX 5070 TiNVIDIA	16GB	896 GB/s	~125 tok/s	Phi-4 Q5_K_M	~$1,200Check Amazon ↗	$75	300W
RTX 4080 SuperNVIDIA	16GB	736 GB/s	~105 tok/s	Phi-4 Q5_K_M	~$1,150 usedCheck Amazon ↗	$72	320W
RTX 4070 Ti SuperNVIDIA	16GB	672 GB/s	~99 tok/s	Phi-4 Q5_K_M	~$900 usedCheck Amazon ↗	$56	285W
RX 9070 XTAMD	16GB	640 GB/s	~75 tok/s	Phi-4 Q5_K_M	~$800Check Amazon ↗	$50	304W
RTX 5060 Ti 16GBNVIDIA	16GB	448 GB/s	~68 tok/s	Phi-4 Q5_K_M	~$620Check Amazon ↗	$39	180W
RTX 4060 Ti 16GBNVIDIA	16GB	288 GB/s	~45 tok/s	Phi-4 Q5_K_M	~$380 usedCheck Amazon ↗	$24	165W
RX 9060 XT 16GBAMD	16GB	320 GB/s	~40 tok/s	Phi-4 Q5_K_M	~$470Check Amazon ↗	$29	160W
RTX 5070NVIDIA	12GB	672 GB/s	~99 tok/s	Phi-4 Q3_K_M	~$800Check Amazon ↗	$67	250W
RTX 4070NVIDIA	12GB	504 GB/s	~76 tok/s	Phi-4 Q3_K_M	~$560 usedCheck Amazon ↗	$47	200W
RTX 3060 12GBNVIDIA	12GB	360 GB/s	~50 tok/s	Phi-4 Q3_K_M	~$270 usedCheck Amazon ↗	$23	170W
Arc B580Intel	12GB	456 GB/s	~46 tok/s	Phi-4 Q3_K_M	~$330Check Amazon ↗	$28	190W
RTX 5060NVIDIA	8GB	448 GB/s	~68 tok/s	Gemma 4 E4B Q4_K_M	~$450Check Amazon ↗	$56	145W

* Unified memory is shared with the CPU; by default the GPU can use roughly 65–75% of it. Prices are typical street prices when last checked and change often.

Frequently asked questions

What matters most in a GPU for running LLMs?

Memory capacity first, then memory bandwidth. Capacity decides which models fit; bandwidth decides how fast they generate text, because every generated token reads the whole active model from memory. Compute (TFLOPS) mostly affects prompt processing.

How much VRAM do I need for local AI?

12GB runs 8B-class models well. 16GB handles 12–14B models and small MoE models. 24GB is the sweet spot for 27–32B models. For 70B dense models you want 48GB or more, either across two GPUs or on a unified-memory system.

Is AMD or Intel good for running LLMs?

Yes, with caveats. AMD Radeon cards work well in llama.cpp (Vulkan or ROCm), LM Studio, and Ollama, and often cost less per GB of VRAM. Intel Arc works through llama.cpp (SYCL or Vulkan) and IPEX-LLM. NVIDIA still has the broadest, least-friction software support.