Skip to content
Llama Builds

Google · Released April 2026

Gemma 4 26B A4B

What it takes to run Gemma 4 26B A4B locally: memory by quantization, the GPUs that handle it, and how fast it runs.

Updated

Parameters
26B (3.8B active)
Architecture
Mixture of experts
Context
256K tokens
License
Apache 2.0 license
Memory at Q4_K_M
~17GB

What do you need to run Gemma 4 26B A4B?

Gemma 4 26B A4B needs about 17GB of VRAM at Q4_K_M with 8K context (16GB for the weights plus KV cache and overhead). The smallest GPU that can hold it is a 16GB card like the RTX 4060 Ti 16GB at Q3_K_M, with little memory to spare. For a good balance of price and speed we recommend the RTX 3090 (~110 tokens/sec). Because it is a mixture-of-experts model with 3.8B active parameters, it generates text much faster than a dense model of the same size, and it tolerates partial CPU offload better.

See the full hardware guide for Gemma 4 26B A4B

What to buy

GPUs for Gemma 4 26B A4B

Cheapest that runs it

RTX 4060 Ti 16GB

16GB · ~$380 used

Quant
Q3_K_M
Speed (est.)
~63 tok/s
Fits, tight
Check price at Amazon

Best value

RTX 3090

24GB · ~$1,050 used

Quant
Q5_K_M
Speed (est.)
~110 tok/s
Runs well
Check price at Amazon

Fastest

RTX 5090

32GB · ~$4,300

Quant
Q6_K
Speed (est.)
~165 tok/s
Runs well
Check price at Amazon

Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.

Memory

Gemma 4 26B A4B VRAM requirements by quantization

Lower-bit quantizations shrink the model at some cost to quality. Q4_K_M is the usual sweet spot. Totals include the KV cache for the context length shown plus runtime overhead.

Memory needed for Gemma 4 26B A4B at each quantization level
QuantizationQualityWeightsTotal @ 8K ctxTotal @ 32K ctx
Q2_KRough9.8 GB11 GB12 GB
Q3_K_MUsable13 GB14 GB15 GB
Q4_K_MRecommendedDefault16 GB17 GB18 GB
Q5_K_MVery good19 GB20 GB21 GB
Q6_KNear-lossless21 GB23 GB24 GB
Q8_0Practically lossless28 GB30 GB30 GB
BF16Full precision52 GB55 GB55 GB

KV cache note: 30 layers in a 5:1 pattern of 1,024-token sliding-window and global layers. Only the global layers grow with context; assuming 5 global layers with 2 KV heads of 512 dims: 2 x 5 x 2 x 512 x 2 bytes = 20,480 bytes per token, about 5.4GB at the full 256K. The sliding layers add a fixed buffer of a few hundred MB.

Performance

How fast Gemma 4 26B A4B runs

Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.

  1. RTX 5090

    Q6_K · 32GB

    ~165tok/s

  2. RTX 5080

    Q3_K_M · 16GB

    ~155tok/s

  3. RTX 5070 Ti

    Q3_K_M · 16GB

    ~145tok/s

  4. RTX PRO 6000

    Q8_0 · 96GB

    ~140tok/s

  5. RTX 4080 Super

    Q3_K_M · 16GB

    ~130tok/s

  6. RTX 4090

    Q5_K_M · 24GB

    ~125tok/s

  7. RTX 4070 Ti Super

    Q3_K_M · 16GB

    ~120tok/s

  8. RTX PRO 5000 72GB

    Q8_0 · 72GB

    ~115tok/s

  9. RTX 3090

    Q5_K_M · 24GB

    ~110tok/s

  10. RX 7900 XTX

    Q5_K_M · 24GB

    ~91tok/s

Estimated (method)

Overview

About Gemma 4 26B A4B

Gemma 4 26B A4B is Google's mixture-of-experts Gemma, with 26B total parameters and 3.8B active (128 experts, 8 used per token), image input, a 256K context, and a thinking mode. Because so few parameters are active, it generates text at small-model speed while matching much larger dense models on reasoning benchmarks. At Q4_K_M it needs about 17GB with 8K context, so it fits a 24GB GPU with plenty of room for context; on 16GB cards it runs at Q3_K_M or with a few experts in system RAM. It replaces Gemma 3 27B for most local uses.

Strengths

  • +Small-model speed: 3.8B active parameters per token
  • +Image input, thinking mode, and 256K context
  • +Compact KV cache thanks to sliding-window attention
  • +Apache 2.0 license

Weaknesses

  • −Slightly too large for 16GB cards at Q4_K_M once context is added
  • −Behind Gemma 4 31B and Qwen3.8 27B on hard reasoning and coding
  • −Less world knowledge than a dense model of the same total size
ChatVisionReasoningCodingAgents & toolsMultilingualImage inputToggleable reasoning

Setup

How to run Gemma 4 26B A4B

Ollama
$ ollama run gemma4:26b
llama.cpp
$ llama-server -hf unsloth/gemma-4-26B-A4B-it-GGUF:Q4_K_M -c 8192 -ngl 99

Weights: google/gemma-4-26B-A4B-it · GGUF: unsloth/gemma-4-26B-A4B-it-GGUF

Alternatives

Models like Gemma 4 26B A4B

Alibaba Qwen

Qwen3.6 35B-A3B

35B (3B active) · 256K ctx

~24GBat Q4_K_M
CodingAgents & toolsReasoning

Google

Gemma 4 31B

31B · 256K ctx

~22GBat Q4_K_M
ChatVisionReasoning

OpenAI

gpt-oss-20b

20.9B (3.6B active) · 128K ctx

~15GBat MXFP4
ReasoningCodingAgents & tools

Alibaba Qwen

Qwen3.5 9B

9B · 256K ctx

~7.2GBat Q4_K_M
ChatReasoningCoding

Frequently asked questions

How much VRAM do you need to run Gemma 4 26B A4B?

About 17GB at Q4_K_M with 8K tokens of context: 15.8GB of weights, 0.5GB of KV cache, and 1.1GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.

What is the cheapest GPU that can run Gemma 4 26B A4B?

The smallest card in our database that can hold it is the RTX 4060 Ti 16GB (16GB), at Q3_K_M with little memory to spare, generating ~63 tokens/sec.

Is Gemma 4 26B A4B free for commercial use?

Gemma 4 26B A4B is released under the Apache 2.0 license. That is a permissive license, so commercial use is allowed.

How long a context does Gemma 4 26B A4B support?

Up to 262,144 tokens (256K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.

Sources: Unsloth Gemma 4 26B A4B GGUF files, Artificial Analysis: Gemma 4 26B A4B vs Qwen3.5 9B, modelfit.io Gemma 4 26B-A4B memory requirements. Capability score basis: Artificial Analysis Intelligence Index v4: 31 in reasoning mode (April 2026), mapped onto the Llama Builds scale just below gpt-oss-120b.