Google · Released April 2026
Gemma 4 E4B
What it takes to run Gemma 4 E4B locally: memory by quantization, the GPUs that handle it, and how fast it runs.
Updated
- Parameters
- 8B
- Architecture
- Dense
- Context
- 128K tokens
- License
- Apache 2.0 license
- Memory at Q4_K_M
- ~6.3GB
What do you need to run Gemma 4 E4B?
Gemma 4 E4B needs about 6.3GB of VRAM at Q4_K_M with 8K context (5.4GB for the weights plus KV cache and overhead). The smallest GPU that runs it comfortably is a 8GB card like the RTX 5060.
What to buy
GPUs for Gemma 4 E4B
Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.
Memory
Gemma 4 E4B VRAM requirements by quantization
Lower-bit quantizations shrink the model at some cost to quality. Q4_K_M is the usual sweet spot. Totals include the KV cache for the context length shown plus runtime overhead.
| Quantization | Quality | Weights | Total @ 8K ctx | Total @ 32K ctx |
|---|---|---|---|---|
| Q2_K | Rough | 3.3 GB | 4.2 GB | 4.6 GB |
| Q3_K_M | Usable | 4.3 GB | 5.2 GB | 5.6 GB |
| Q4_K_M | RecommendedDefault | 5.4 GB | 6.3 GB | 6.7 GB |
| Q5_K_M | Very good | 6.3 GB | 7.3 GB | 7.7 GB |
| Q6_K | Near-lossless | 7.3 GB | 8.3 GB | 8.7 GB |
| Q8_0 | Practically lossless | 9.5 GB | 10 GB | 11 GB |
| BF16 | Full precision | 18 GB | 19 GB | 19 GB |
KV cache note: 42 layers, but 18 reuse the KV cache of earlier layers and most of the rest use a 512-token sliding window. Only 4 global layers grow with context: 2 x 4 x 2 KV heads x 512 dims x 2 bytes = 16,384 bytes per token (llama.cpp reports 2,048 MiB for them at 131,072 tokens). The sliding layers add a small fixed buffer.
Performance
How fast Gemma 4 E4B runs
Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.
RTX 5090
Q8_0 · 32GB
~130tok/s
RTX PRO 6000
Q8_0 · 96GB
~130tok/s
RTX PRO 5000 72GB
Q8_0 · 72GB
~100tok/s
RTX 4090
Q8_0 · 24GB
~79tok/s
RTX 5080
Q8_0 · 16GB
~75tok/s
RTX 6000 Ada
Q8_0 · 48GB
~75tok/s
RTX 5070 Ti
Q8_0 · 16GB
~70tok/s
RTX 3090
Q8_0 · 24GB
~66tok/s
M5 Ultra 512GB
Q8_0 · 512GB
~64tok/s
M5 Ultra 256GB
Q8_0 · 256GB
~64tok/s
Overview
About Gemma 4 E4B
Gemma 4 E4B is the larger of Google's two on-device Gemma 4 models: about 4.5B effective parameters (8B including per-layer embeddings) with text, image, and audio input and an optional thinking mode. It replaces Gemma 3n and the small Gemma 3 models, and it ships under Apache 2.0 instead of the old Gemma license. At Q4_K_M it needs about 6GB with 8K context, so it runs on any 8GB GPU, on 16GB laptops and Macs, and even on CPU-only machines at usable speed. Its KV cache is tiny, so the full 128K context adds only about 2GB.
Strengths
- +Text, image, and audio input in a model that fits 8GB
- +Very small KV cache: the full 128K context adds about 2GB
- +Optional thinking mode for harder questions
- +Apache 2.0 license
Weaknesses
- −Limited knowledge and reasoning depth compared with 9B to 12B models
- −Per-layer embeddings make the file larger than the 4.5B effective size suggests
- −Long thinking traces are slow on CPU-only machines
Setup
How to run Gemma 4 E4B
$ ollama run gemma4:e4b$ llama-server -hf unsloth/gemma-4-E4B-it-GGUF:Q4_K_M -c 8192 -ngl 99Weights: google/gemma-4-E4B-it · GGUF: unsloth/gemma-4-E4B-it-GGUF
Alternatives
Models like Gemma 4 E4B
Frequently asked questions
How much VRAM do you need to run Gemma 4 E4B?
About 6.3GB at Q4_K_M with 8K tokens of context: 5.4GB of weights, 0.1GB of KV cache, and 0.8GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.
What is the cheapest GPU that can run Gemma 4 E4B?
The smallest card in our database that runs it comfortably is the RTX 5060 (8GB), at Q4_K_M and ~62 tokens/sec.
Is Gemma 4 E4B free for commercial use?
Gemma 4 E4B is released under the Apache 2.0 license. That is a permissive license, so commercial use is allowed.
How long a context does Gemma 4 E4B support?
Up to 131,072 tokens (128K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.
Sources: Unsloth Gemma 4 E4B GGUF files, modelfit.io GGUF file sizes for Gemma 4 E4B, Ollama gemma4 library. Capability score basis: Estimated from its position below Gemma 4 26B A4B (31) on the Artificial Analysis Intelligence Index v4 (April 2026); no direct E4B reading was used.