Microsoft · Released December 2024
Phi-4
What it takes to run Phi-4 locally: memory by quantization, the GPUs that handle it, and how fast it runs.
Updated
- Parameters
- 14.7B
- Architecture
- Dense
- Context
- 16K tokens
- License
- MIT license
- Memory at Q4_K_M
- ~12GB
What do you need to run Phi-4?
Phi-4 needs about 12GB of VRAM at Q4_K_M with 8K context (9.1GB for the weights plus KV cache and overhead). The smallest GPU that runs it comfortably at Q3_K_M is a 12GB card like the RTX 3060 12GB.
What to buy
GPUs for Phi-4
Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.
Memory
Phi-4 VRAM requirements by quantization
Lower-bit quantizations shrink the model at some cost to quality. Q4_K_M is the usual sweet spot. Totals include the KV cache for the context length shown plus runtime overhead.
| Quantization | Quality | Weights | Total @ 8K ctx |
|---|---|---|---|
| Q2_K | Rough | 5.6 GB | 8.0 GB |
| Q3_K_M | Usable | 7.3 GB | 9.8 GB |
| Q4_K_M | RecommendedDefault | 9.1 GB | 12 GB |
| Q5_K_M | Very good | 11 GB | 13 GB |
| Q6_K | Near-lossless | 12 GB | 15 GB |
| Q8_0 | Practically lossless | 16 GB | 18 GB |
| BF16 | Full precision | 29 GB | 32 GB |
KV cache note: 2 x 40 layers x 10 KV heads x 128 dims x 2 bytes = 204,800 bytes (200 KiB) per token, about 1.7GB at 8K and 3.4GB at the full 16K context.
Performance
How fast Phi-4 runs
Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.
RTX 5090
Q8_0 · 32GB
~84tok/s
RTX PRO 6000
Q8_0 · 96GB
~84tok/s
RTX 5070
Q3_K_M · 12GB
~69tok/s
RTX 5080
Q5_K_M · 16GB
~67tok/s
RTX PRO 5000 72GB
Q8_0 · 72GB
~65tok/s
RTX 5070 Ti
Q5_K_M · 16GB
~63tok/s
RTX 4080 Super
Q5_K_M · 16GB
~52tok/s
RTX 4090
Q8_0 · 24GB
~49tok/s
RTX 4070 Ti Super
Q5_K_M · 16GB
~48tok/s
RTX 6000 Ada
Q8_0 · 48GB
~47tok/s
Overview
About Phi-4
Phi-4 is Microsoft's 14B dense model, trained heavily on synthetic data to punch above its size on math and reasoning. Microsoft has not released a Phi-5, so it remains the current base model of the Phi line, alongside reasoning-tuned variants such as Phi-4-reasoning. At Q4_K_M it needs about 12GB with 8K context, which is tight on a 12GB card and comfortable on 16GB. Its 16K context limit and narrow knowledge make Qwen3.5 9B or gpt-oss-20b better general picks in 2026.
Strengths
- +Strong math and logic for a 14B non-reasoning model
- +Dense and simple to run in every runtime
- +MIT license
Weaknesses
- −16K context window
- −Narrow world knowledge and weak creative writing
- −Text only, with no reasoning mode in the base model
- −Outclassed by smaller 2026 models such as Qwen3.5 9B
Setup
How to run Phi-4
$ ollama run phi4:14b$ llama-server -hf bartowski/phi-4-GGUF:Q4_K_M -c 8192 -ngl 99$ mlx_lm.chat --model mlx-community/phi-4-4bitWeights: microsoft/phi-4 · GGUF: bartowski/phi-4-GGUF
Alternatives
Models like Phi-4
Frequently asked questions
How much VRAM do you need to run Phi-4?
About 12GB at Q4_K_M with 8K tokens of context: 9.1GB of weights, 1.7GB of KV cache, and 0.9GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.
What is the cheapest GPU that can run Phi-4?
The smallest card in our database that runs it comfortably is the RTX 3060 12GB (12GB), at Q3_K_M and ~34 tokens/sec.
Is Phi-4 free for commercial use?
Phi-4 is released under the MIT license. That is a permissive license, so commercial use is allowed.
How long a context does Phi-4 support?
Up to 16,384 tokens (16K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.
Sources: Phi-4 model card (Hugging Face), Phi-4 technical report. Capability score basis: Placed between the Llama 3.1 8B and Qwen3 32B anchors from its Artificial Analysis Intelligence Index v3 standing (October 2025).