Skip to content
Llama Builds

Alibaba Qwen · Released March 2026

Qwen3.5 9B

What it takes to run Qwen3.5 9B locally: memory by quantization, the GPUs that handle it, and how fast it runs.

Updated

Parameters
9B
Architecture
Dense
Context
256K tokens
License
Apache 2.0 license
Memory at Q4_K_M
~7.2GB

What do you need to run Qwen3.5 9B?

Qwen3.5 9B needs about 7.2GB of VRAM at Q4_K_M with 8K context (6.2GB for the weights plus KV cache and overhead). The smallest GPU that can hold it is a 8GB card like the RTX 5060, with little memory to spare. For a good balance of price and speed we recommend the RTX 3060 12GB (~30 tokens/sec).

See the full hardware guide for Qwen3.5 9B

What to buy

GPUs for Qwen3.5 9B

Cheapest that runs it

RTX 5060

8GB · ~$450

Quant
Q4_K_M
Speed (est.)
~55 tok/s
Fits, tight
Check price at Amazon

Best value

RTX 3060 12GB

12GB · ~$270 used

Quant
Q6_K
Speed (est.)
~30 tok/s
Runs well
Check price at Amazon

Fastest

RTX 5090

32GB · ~$4,300

Quant
Q8_0
Speed (est.)
~115 tok/s
Runs well
Check price at Amazon

Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.

Memory

Qwen3.5 9B VRAM requirements by quantization

Lower-bit quantizations shrink the model at some cost to quality. Q4_K_M is the usual sweet spot. Totals include the KV cache for the context length shown plus runtime overhead.

Memory needed for Qwen3.5 9B at each quantization level
QuantizationQualityWeightsTotal @ 8K ctxTotal @ 32K ctx
Q2_KRough3.8 GB4.8 GB5.6 GB
Q3_K_MUsable5.0 GB6.0 GB6.8 GB
Q4_K_MRecommendedDefault6.2 GB7.2 GB8.0 GB
Q5_K_MVery good7.3 GB8.3 GB9.1 GB
Q6_KNear-lossless8.4 GB9.5 GB10 GB
Q8_0Practically lossless11 GB12 GB13 GB
BF16Full precision20 GB22 GB23 GB

KV cache note: Hybrid layout: three of every four layers use Gated DeltaNet linear attention with a fixed-size state, and only the full-attention layers keep a growing cache. Assuming 8 full-attention layers with 4 KV heads of 256 dims: 2 x 8 x 4 x 256 x 2 bytes = 32,768 bytes per token, about 8.6GB at the full 262K. The linear layers add a small fixed state.

Performance

How fast Qwen3.5 9B runs

Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.

  1. RTX 5090

    Q8_0 · 32GB

    ~115tok/s

  2. RTX PRO 6000

    Q8_0 · 96GB

    ~115tok/s

  3. RTX PRO 5000 72GB

    Q8_0 · 72GB

    ~90tok/s

  4. RTX 4090

    Q8_0 · 24GB

    ~69tok/s

  5. RTX 5080

    Q8_0 · 16GB

    ~66tok/s

  6. RTX 6000 Ada

    Q8_0 · 48GB

    ~66tok/s

  7. RTX 5070 Ti

    Q8_0 · 16GB

    ~62tok/s

  8. RTX 5070

    Q6_K · 12GB

    ~60tok/s

  9. RTX 3090

    Q8_0 · 24GB

    ~59tok/s

  10. M5 Ultra 512GB

    Q8_0 · 512GB

    ~57tok/s

Estimated (method)

Overview

About Qwen3.5 9B

Qwen3.5 9B is the largest of the small Qwen3.5 models released in March 2026, with built-in image understanding, a 262K context, and switchable thinking. It uses Qwen's hybrid attention, where most layers are linear-attention layers with a fixed-size state, so long contexts cost far less memory than on older 8B models. It is one of the strongest models that fits an 8GB GPU: at Q4_K_M it needs about 7GB with 8K context, and 12GB cards can run Q8_0. It replaces Qwen3 8B and, for most uses, Qwen3 14B.

Strengths

  • +Image input, toggleable thinking, and 262K context in an 8GB-class model
  • +Hybrid attention keeps long-context memory low
  • +Strong reasoning and coding for its size
  • +Apache 2.0 license

Weaknesses

  • −Thinking mode is verbose, so answers take longer on slow hardware
  • −Limited world knowledge, as with any 9B model
  • −Linear-attention layers need recent runtime versions for full speed
ChatReasoningCodingVisionMultilingualAgents & toolsImage inputToggleable reasoning

Setup

How to run Qwen3.5 9B

Ollama
$ ollama run qwen3.5:9b
llama.cpp
$ llama-server -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M -c 8192 -ngl 99

Weights: Qwen/Qwen3.5-9B · GGUF: unsloth/Qwen3.5-9B-GGUF

Alternatives

Models like Qwen3.5 9B

Google

Gemma 4 E4B

8B · 128K ctx

~6.3GBat Q4_K_M
ChatVisionMultilingual

Alibaba Qwen

Qwen3 8B

8.2B · 32K ctx

~7GBat Q4_K_M
ChatReasoningCoding

OpenAI

gpt-oss-20b

20.9B (3.6B active) · 128K ctx

~15GBat MXFP4
ReasoningCodingAgents & tools

Alibaba Qwen

Qwen3.8 27B

27B · 256K ctx

~18GBat Q4_K_M
CodingAgents & toolsReasoning

Frequently asked questions

How much VRAM do you need to run Qwen3.5 9B?

About 7.2GB at Q4_K_M with 8K tokens of context: 6.2GB of weights, 0.3GB of KV cache, and 0.8GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.

What is the cheapest GPU that can run Qwen3.5 9B?

The smallest card in our database that can hold it is the RTX 5060 (8GB), at Q4_K_M with little memory to spare, generating ~55 tokens/sec.

Is Qwen3.5 9B free for commercial use?

Qwen3.5 9B is released under the Apache 2.0 license. That is a permissive license, so commercial use is allowed.

How long a context does Qwen3.5 9B support?

Up to 262,144 tokens (256K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.

Sources: Qwen3.5 9B model card (Hugging Face), modelfit.io GGUF file sizes for Qwen3.5 9B, Artificial Analysis: Qwen3.5 small models. Capability score basis: Estimated from Artificial Analysis Intelligence Index v4 coverage of the Qwen3.5 small models (March 2026) and placed level with gpt-oss-20b; no exact reading was confirmed.