NVIDIA · Ada Lovelace · 2023
NVIDIA GeForce RTX 4070 for local AI
An efficient 12GB card for small models, outclassed by 16GB options at similar prices.
Updated · Speeds marked ~ are estimates; seemethodology
- VRAM
- 12GB
- Bandwidth
- 504 GB/s
- Power
- 200W
- Runs well
- 3 models
RTX 4070
Typically about $560 used
Can the RTX 4070 run LLMs?
The RTX 4070 has 12GB of GDDR6X and 504 GB/s of memory bandwidth, which decides how big a model fits and how fast it generates text. It comfortably runs models up to about 15B parameters: the largest in our catalog is Phi-4 at Q3_K_M (9.8GB with 8K context, ~52 tokens/sec). Our top all-round pick for it is Qwen3.5 9B (Q6_K, ~46 tokens/sec). Llama 3.3 70B Instruct only runs with partial CPU offload, at about 1.2 tokens/sec with CPU offload.
Recommendations
Best models for the RTX 4070
Best overall · also coding, reasoning, vision, fastest
Qwen3.5 9BQ6_K · 10GB · ~46 tokens/sec
Runs wellLargest model that fits
Phi-4Q3_K_M · 10GB · ~52 tokens/sec
Runs wellStretch pick (CPU offload)
Qwen3.6 35B-A3BQ4_K_M · 24GB · about 34 tokens/sec with CPU offload
Partial offload
Compatibility
What runs on the RTX 4070
Every current model in our catalog, at the best quantization that fits in 12GB with 8K tokens of context. "Partial offload" means part of the model runs from system RAM (we assume 64GB), which is much slower.
| Model | Parameters | Best quant | Memory | Speed | Verdict |
|---|---|---|---|---|---|
| Gemma 4 E4B | 8B | Q8_0 | 10 GB | ~41 tok/s | Runs well |
| Qwen3.5 9B | 9B | Q6_K | 9.5 GB | ~46 tok/s | Runs well |
| Phi-4 | 14.7B | Q3_K_M | 9.8 GB | ~52 tok/s | Runs well |
| gpt-oss-20b | 20.9B (3.6B active) | MXFP4 | 15 GB | ~43 tok/s | Partial offload |
| Gemma 4 26B A4B | 26B (3.8B active) | Q4_K_M | 17 GB | ~37 tok/s | Partial offload |
| Qwen3.8 27B | 27B | Q4_K_M | 18 GB | ~5.8 tok/s | Partial offload |
| Gemma 4 31B | 31B | Q4_K_M | 22 GB | ~3.8 tok/s | Partial offload |
| Qwen3.6 35B-A3B | 35B (3B active) | Q4_K_M | 24 GB | ~34 tok/s | Partial offload |
| GLM-4.5-Air | 106B (12B active) | — | — | — | Won't fitWhat it needs |
| Llama 4 Scout | 109B (17B active) | — | — | — | Won't fitWhat it needs |
| gpt-oss-120b | 117B (5.1B active) | — | — | — | Won't fitWhat it needs |
| Qwen3.5 122B-A10B | 122B (10B active) | — | — | — | Won't fitWhat it needs |
| DeepSeek V4 Flash | 284B (13B active) | — | — | — | Won't fitWhat it needs |
| GLM-5.3-Flash | 320B (18B active) | — | — | — | Won't fitWhat it needs |
| GLM-5.3 | 744B (40B active) | — | — | — | Won't fitWhat it needs |
Measured
RTX 4070 benchmark results
| Model | Quant | Runtime | Generation | Prompt | Source |
|---|---|---|---|---|---|
| Llama 2 7B Chat | Q4_0 | llama.cpp (Vulkan) | 92 tok/s | 3180 tok/s | llama.cpp discussion #10879: Performance of llama.cpp with Vulkan · October 2026 |
Our take
Should you buy the RTX 4070 for AI?
The RTX 4070 has 12GB of GDDR6X on a 192-bit bus for 504 GB/s and draws only about 200W. It runs 8B models quickly and 14B models at 4 to 6 bit with short to moderate context. NVIDIA later added a GDDR6 version at 480 GB/s, which is slightly slower. Used cards sold for about $560 in October 2026, close to the price of a new RTX 5060 Ti 16GB, which holds larger models.
What we like
- Low 200W power draw
- Compact cards fit small cases
- Good speed on 8B to 14B models
Watch out for
- 12GB limits model size and context
- Used price close to a new 16GB card
Best for: Small form factor builds · Fast 8B to 14B assistants
Specifications
RTX 4070 specs that matter for AI
- Memory
- 12GB GDDR6X
- Memory bandwidth
- 504 GB/s
- Memory bus
- 192-bit
- Architecture
- Ada Lovelace
- Power
- 200W
- Interface
- PCIe 4.0 x16
- Released
- 2023
- Launch price
- $599
- Street price (used)
- ~$560 (2026-10)
- Software
- CUDA, Vulkan
A later GDDR6 version runs at 480 GB/s.
Builds
PC builds with the RTX 4070
Cross-shop
Alternatives to the RTX 4070
Frequently asked questions
How much VRAM does the RTX 4070 have?
The NVIDIA GeForce RTX 4070 has 12GB of GDDR6X with 504 GB/s of memory bandwidth on a 192-bit bus.
What is the largest LLM the RTX 4070 can run?
Fully in GPU memory, the largest model in our catalog that runs comfortably is Phi-4 (14.7B) at Q3_K_M, using about 10GB with 8K context at ~52 tokens/sec.
Can the RTX 4070 run Llama 3.3 70B?
Only with partial CPU offload. At Q4_K_M the model needs about 47GB with 8K context, so part of it spills into system RAM and speed drops to about 1.2 tokens/sec with CPU offload. For usable 70B speeds you want 48GB of VRAM or more (for example two 24GB cards).
What is the best LLM to run on the RTX 4070?
Our top pick is Qwen3.5 9B at Q6_K, which fits in 12GB with room for 8K of context and runs at ~46 tokens/sec. See our full list of the best models for the RTX 4070 for coding, reasoning, and vision picks.
Is the RTX 4070 good for local AI in 2026?
An efficient 12GB card for small models, outclassed by 16GB options at similar prices.