The best local LLMs for the RTX 5070

Qwen3.5 9B (9B) fits comfortably in the RTX 5070's memory at Q6_K, using about 9.5GB with 8K context and generating ~60 tokens/sec. It is the most capable model in our catalog that runs well on this card. It is also our top pick for coding, reasoning, working with images and speed.

Qwen3.5 9B is the largest of the small Qwen3.5 models released in March 2026, with built-in image understanding, a 262K context, and switchable thinking. It uses Qwen's hybrid attention, where most layers are linear-attention layers with a fixed-size state, so long contexts cost far less memory than on older 8B models. It is one of the strongest models that fits an 8GB GPU: at Q4_K_M it needs about 7GB with 8K context, and 12GB cards can run Q8_0. It replaces Qwen3 8B and, for most uses, Qwen3 14B.

Phi-4 (14.7B) fits comfortably in the RTX 5070's memory at Q3_K_M, using about 9.8GB with 8K context and generating ~69 tokens/sec. It is the largest model that runs fully on this GPU.

Phi-4 is Microsoft's 14B dense model, trained heavily on synthetic data to punch above its size on math and reasoning. Microsoft has not released a Phi-5, so it remains the current base model of the Phi line, alongside reasoning-tuned variants such as Phi-4-reasoning. At Q4_K_M it needs about 12GB with 8K context, which is tight on a 12GB card and comfortable on 16GB. Its 16K context limit and narrow knowledge make Qwen3.5 9B or gpt-oss-20b better general picks in 2026.

Qwen3.6 35B-A3B (35B (3B active)) runs with part of the model in system RAM in the RTX 5070's memory at Q4_K_M, using about 24GB with 8K context and generating about 35 tokens/sec with CPU offload. Because it is a mixture-of-experts model, only 3B parameters are active per token, so offloading experts to system RAM still gives usable speed (assumes 64GB of system RAM).

Qwen3.6 35B-A3B is the April 2026 refresh of Qwen's fast mixture-of-experts model, focused on agentic coding and on preserving its reasoning across multi-turn tool use. With only 3B active parameters it generates text very quickly, and its hybrid linear attention keeps the KV cache tiny: the full 262K context needs about 5GB. At Q4_K_M it needs about 24GB with 8K context, which is tight on a 24GB card and comfortable on 32GB GPUs and Macs with 36GB or more; 12GB to 16GB GPUs run it well with experts offloaded to system RAM. It replaces Qwen3 30B-A3B and Qwen3 Coder 30B-A3B for most local users.