Gemma 4 26B A4B IT vs Qwen3.5 9B
Gemma 4 26B A4B IT leads llmrun Score 105.4 to 99.4 (ranges overlap); both fit 24 GB at Q4_K_M; Gemma 4 26B A4B IT is 1.7× faster on RTX 4090 (~171 vs ~103 tok/s).
Model A · Google
Gemma 4 26B A4B IT25.8B · 4.4B active · 256K context · Apache 2.0 · Mar 2026
Model B · Alibaba
Qwen3.5 9B9.7B · 256K context · Apache 2.0 · Feb 2026
Choose Gemma 4 26B A4B IT if you want…
Higher llmrun Score
105.4 vs 99.4, ranges overlap
1.7× faster on RTX 4090
~171 vs ~103 tok/s
Choose Qwen3.5 9B if you want…
Fits an 8 GB GPU
Q4_K_M · 6.6 GB; Gemma 4 26B A4B IT needs 8.1 GB at its smallest quant
Lighter on memory
6.4 vs 16.1 GB at Q4_K_M
On your hardware
Each model at its default quant (Q4_K_M), a smaller one when it must. Estimates, ±20%.
8 GB
RTX 4060 · 8 GB tier
Gemma 4 26B A4B IT
Doesn't fit
smallest quant needs 8.1 GB
Qwen3.5 9B
Q4_K_M · ~28 tok/s
6.6 GB · up to 32Kfits
12 GB
RTX 3060 12GB · 12 GB tier
Gemma 4 26B A4B IT
IQ3_S · ~99 tok/s
12 GB · up to 16K
Qwen3.5 9B
Q4_K_M · ~37 tok/s
6.6 GB · up to 128Kmore headroom
16 GB
RTX 5060 Ti 16GB · 16 GB tier
Gemma 4 26B A4B IT
Q4_1 · ~95 tok/s
15.5 GB · up to 16K
Qwen3.5 9B
Q4_K_M · ~46 tok/s
6.6 GB · up to 256Kmore headroom
24 GBreference
RTX 4090 · 24 GB tier
Gemma 4 26B A4B IT
Q4_K_M · ~171 tok/s
16.5 GB · up to 128Kfaster
Qwen3.5 9B
Q4_K_M · ~103 tok/s
6.6 GB · up to 256K
32 GB
RTX 5090 · 32 GB tier
Gemma 4 26B A4B IT
Q4_K_M · ~250 tok/s
16.5 GB · up to 256Kfaster
Qwen3.5 9B
Q4_K_M · ~183 tok/s
6.6 GB · up to 256K
48 GB
RTX A6000 · 48 GB tier
Gemma 4 26B A4B IT
Q4_K_M · ~140 tok/s
16.5 GB · up to 256Kfaster
Qwen3.5 9B
Q4_K_M · ~79 tok/s
6.6 GB · up to 256K
Mac 32 GB
Mac Mini M4 · Macs
Gemma 4 26B A4B IT
Q4_K_M · ~26 tok/s
16.5 GB · up to 256Kfaster
Qwen3.5 9B
Q4_K_M · ~13 tok/s
6.6 GB · up to 256K
Mac 64 GB
Mac Studio M4 Max · Macs
Gemma 4 26B A4B IT
Q4_K_M · ~76 tok/s
16.5 GB · up to 256Kfaster
Qwen3.5 9B
Q4_K_M · ~60 tok/s
6.6 GB · up to 256K
Mac 128 GB
Mac Studio M4 Max · Macs
Gemma 4 26B A4B IT
Q4_K_M · ~76 tok/s
16.5 GB · up to 256Kfaster
Qwen3.5 9B
Q4_K_M · ~60 tok/s
6.6 GB · up to 256K
Reference GPU: RTX 4090. Tinted cell = the better pick on that machine (fits, more headroom, or at least 10% faster).
Overall
Gemma 4 26B A4B IT
Qwen3.5 9B
Coding
Gemma 4 26B A4B IT
Qwen3.5 9B
no data
Science
Gemma 4 26B A4B IT
Qwen3.5 9B
Reasoning
Gemma 4 26B A4B IT
Qwen3.5 9B
Gemma 4 26B A4B IT · #37 of 92 open models · #6 of 34 that fit 24 GB
Qwen3.5 9B · #49 of 92 open models · #10 of 34 that fit 24 GB
Benchmarks
7 shared boards: Gemma 4 26B A4B IT leads 4, Qwen3.5 9B leads 3 · Benchmarks imported Oct 2026
Gemma 4 26B A4B IT
73.2%#29/97 open
IndependentEpoch AI · Apr 2026
Qwen3.5 9B
79.0%#25/97 open
IndependentEpoch AI · Feb 2026
Gemma 4 26B A4B IT
82.2%#23/85 open
IndependentEpoch AI · Apr 2026
Qwen3.5 9B
61.7%#36/85 open
IndependentEpoch AI · Feb 2026
Gemma 4 26B A4B IT
40.1%#30/65 open
Lab-reportedEpoch AI · Apr 2026
Qwen3.5 9B
27.6%#54/65 open
Lab-reportedEpoch AI · Feb 2026
Gemma 4 26B A4B IT
6.0%#27/67 open
IndependentEpoch AI · Apr 2026
Qwen3.5 9B
12.0%#23/67 open
IndependentEpoch AI · Feb 2026
Gemma 4 26B A4B IT
0.0%#50/65 open
Lab-reportedEpoch AI · Apr 2026
Qwen3.5 9B
0.3%#43/65 open
Lab-reportedEpoch AI · Feb 2026
Gemma 4 26B A4B IT
74.9%#29/68 open
Lab-reportedEpoch AI · Apr 2026
Qwen3.5 9B
71.2%#31/68 open
Lab-reportedEpoch AI · Feb 2026
Gemma 4 26B A4B IT
29.7%#20/54 open
Lab-reportedEpoch AI · Apr 2026
Qwen3.5 9B
24.5%#28/54 open
Lab-reportedEpoch AI · Feb 2026
6 more boards (other shared and single-model boards)
Only Gemma 4 26B A4B IT (4)
- LMArena Text1437.6
- LMArena WebDev1359.0
- WeirdML35.2%
- ALE-Bench927.2
Only Qwen3.5 9B (2)
- Terminal-Bench9.2%
- MMLU-Pro82.5%
VRAM by quantization
Memory to load the model and 2K context; below it, the total at 32K and 128K context. Lighter side in bold.
Q2_K
Gemma 4 26B A4B IT
11.6 GB11.0 GB file
32K 12.5 GB · 128K 15.8 GB
Qwen3.5 9B
4.7 GB4.3 GB file
32K 5.5 GB · 128K 8.7 GB
Q3_K_S
Gemma 4 26B A4B IT
11.9 GB12.6 GB file
32K 12.8 GB · 128K 16.2 GB
Qwen3.5 9B
4.8 GB4.3 GB file
32K 5.6 GB · 128K 8.8 GB
Q3_K_M
Gemma 4 26B A4B IT
13.2 GB12.7 GB file
32K 14.1 GB · 128K 17.5 GB
Qwen3.5 9B
5.3 GB4.7 GB file
32K 6.1 GB · 128K 9.3 GB
Q4_0
Gemma 4 26B A4B IT
13.6 GB~12.9 GB file
32K 14.5 GB · 128K 17.8 GB
Qwen3.5 9B
5.4 GB5.4 GB file
32K 6.2 GB · 128K 9.4 GB
Q4_K_M
Gemma 4 26B A4B IT
16.1 GB16.9 GB file
32K 17.0 GB · 128K 20.4 GB
Qwen3.5 9B
6.4 GB5.7 GB file
32K 7.2 GB · 128K 10.4 GB
Q5_K_M
Gemma 4 26B A4B IT
19.0 GB21.2 GB file
32K 19.9 GB · 128K 23.3 GB
Qwen3.5 9B
7.5 GB6.6 GB file
32K 8.3 GB · 128K 11.5 GB
Q6_K
Gemma 4 26B A4B IT
21.9 GB23.2 GB file
32K 22.8 GB · 128K 26.2 GB
Qwen3.5 9B
8.5 GB7.5 GB file
32K 9.3 GB · 128K 12.6 GB
Q8_0
Gemma 4 26B A4B IT
26.5 GB~25.8 GB file
32K 27.4 GB · 128K 30.7 GB
Qwen3.5 9B
10.2 GB9.5 GB file
32K 11.0 GB · 128K 14.3 GB
19 more quants
IQ2_XXS
Gemma 4 26B A4B IT
7.7 GB9.9 GB file
32K 8.6 GB · 128K 12.0 GB
Qwen3.5 9B
3.2 GB3.2 GB file
32K 4.0 GB · 128K 7.3 GB
IQ2_XS
Gemma 4 26B A4B IT
8.4 GB10.1 GB file
32K 9.3 GB · 128K 12.6 GB
Qwen3.5 9B
not listed
IQ2_S
Gemma 4 26B A4B IT
8.7 GB10.2 GB file
32K 9.6 GB · 128K 12.9 GB
Qwen3.5 9B
3.6 GB3.5 GB file
32K 4.4 GB · 128K 7.6 GB
IQ2_M
Gemma 4 26B A4B IT
9.4 GB10.0 GB file
32K 10.3 GB · 128K 13.6 GB
Qwen3.5 9B
3.8 GB3.6 GB file
32K 4.6 GB · 128K 7.9 GB
IQ3_XXS
Gemma 4 26B A4B IT
10.7 GB11.4 GB file
32K 11.6 GB · 128K 14.9 GB
Qwen3.5 9B
4.3 GB4.0 GB file
32K 5.1 GB · 128K 8.3 GB
Q2_K_S
Gemma 4 26B A4B IT
11.0 GB10.6 GB file
32K 11.9 GB · 128K 15.2 GB
Qwen3.5 9B
not listed
IQ3_XS
Gemma 4 26B A4B IT
11.3 GB12.4 GB file
32K 12.2 GB · 128K 15.5 GB
Qwen3.5 9B
4.6 GB4.8 GB file
32K 5.4 GB · 128K 8.6 GB
IQ3_S
Gemma 4 26B A4B IT
11.6 GB11.3 GB file
32K 12.5 GB · 128K 15.8 GB
Qwen3.5 9B
4.7 GB3.2 GB file
32K 5.5 GB · 128K 8.7 GB
IQ3_M
Gemma 4 26B A4B IT
12.3 GB13.3 GB file
32K 13.2 GB · 128K 16.5 GB
Qwen3.5 9B
4.9 GB5.0 GB file
32K 5.7 GB · 128K 8.9 GB
Q3_K_L
Gemma 4 26B A4B IT
13.9 GB13.2 GB file
32K 14.8 GB · 128K 18.1 GB
Qwen3.5 9B
5.5 GB5.4 GB file
32K 6.3 GB · 128K 9.6 GB
IQ4_XS
Gemma 4 26B A4B IT
14.5 GB13.6 GB file
32K 15.4 GB · 128K 18.7 GB
Qwen3.5 9B
5.8 GB5.2 GB file
32K 6.6 GB · 128K 9.8 GB
IQ4_NL
Gemma 4 26B A4B IT
15.2 GB13.6 GB file
32K 16.1 GB · 128K 19.4 GB
Qwen3.5 9B
6.0 GB5.4 GB file
32K 6.8 GB · 128K 10.0 GB
Q4_1
Gemma 4 26B A4B IT
15.2 GB16.1 GB file
32K 16.1 GB · 128K 19.4 GB
Qwen3.5 9B
6.0 GB5.8 GB file
32K 6.8 GB · 128K 10.0 GB
Q4_K_S
Gemma 4 26B A4B IT
15.2 GB16.5 GB file
32K 16.1 GB · 128K 19.4 GB
Qwen3.5 9B
6.0 GB5.4 GB file
32K 6.8 GB · 128K 10.0 GB
Q4_K_L
Gemma 4 26B A4B IT
16.5 GB17.2 GB file
32K 17.4 GB · 128K 20.7 GB
Qwen3.5 9B
6.5 GB6.9 GB file
32K 7.3 GB · 128K 10.5 GB
Q5_0
Gemma 4 26B A4B IT
16.8 GB17.5 GB file
32K 17.7 GB · 128K 21.0 GB
Qwen3.5 9B
not listed
Q5_K_S
Gemma 4 26B A4B IT
18.4 GB18.9 GB file
32K 19.3 GB · 128K 22.6 GB
Qwen3.5 9B
7.2 GB6.4 GB file
32K 8.0 GB · 128K 11.2 GB
Q5_K_L
Gemma 4 26B A4B IT
19.4 GB19.5 GB file
32K 20.3 GB · 128K 23.6 GB
Qwen3.5 9B
7.6 GB7.7 GB file
32K 8.4 GB · 128K 11.6 GB
BF16
Gemma 4 26B A4B IT
52.3 GB~51.6 GB file
32K 53.2 GB · 128K 56.5 GB
Qwen3.5 9B
19.9 GB17.9 GB file
32K 20.7 GB · 128K 23.9 GB
Gemma 4 26B A4B IT
Ollama · Q4_K_M
ollama run hf.co/unsloth/gemma-4-26B-A4B-it-GGUF:Q4_K_MQwen3.5 9B
Ollama · Q4_K_M
ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q4_K_M| Spec | Gemma 4 26B A4B IT | Qwen3.5 9B |
|---|---|---|
| Publisher | Alibaba | |
| Parameters | 25.8B | 9.7B |
| Active parameters | 4.4B per token | — |
| Architecture | Gemma4ForConditionalGeneration | Qwen3_5ForConditionalGeneration |
| Context | 256K tokens | 256K tokens |
| Modalities | Vision, Function-calling | Vision, Function-calling |
| License | Apache 2.0 | Apache 2.0 |
| Released | Mar 2026 | Feb 2026 |
Frequently Asked Questions
Which is better, Gemma 4 26B A4B IT or Qwen3.5 9B?
Gemma 4 26B A4B IT leads llmrun Score 105.4 to 99.4 (ranges overlap); both fit 24 GB at Q4_K_M; Gemma 4 26B A4B IT is 1.7× faster on RTX 4090 (~171 vs ~103 tok/s). Choose Gemma 4 26B A4B IT for higher llmrun Score and 1.7× faster on RTX 4090; choose Qwen3.5 9B for fits an 8 GB GPU and lighter on memory.
Which needs less VRAM, Gemma 4 26B A4B IT or Qwen3.5 9B?
At Q4_K_M, Gemma 4 26B A4B IT requires 16.1 GB while Qwen3.5 9B needs 6.4 GB, making Qwen3.5 9B the easier model to run on your local setup.
Which is faster on RTX 4090?
Estimated decode speed on RTX 4090 is about 171 tok/s for Gemma 4 26B A4B IT (Q4_K_M) and 103 tok/s for Qwen3.5 9B (Q4_K_M), so Gemma 4 26B A4B IT is faster. Estimates, ±20%.
Can I run Gemma 4 26B A4B IT or Qwen3.5 9B on a 16 GB GPU?
Gemma 4 26B A4B IT fits at Q4_1; Qwen3.5 9B fits at Q4_K_M.
Which has a longer context window, Gemma 4 26B A4B IT or Qwen3.5 9B?
Gemma 4 26B A4B IT supports 262,144 tokens and Qwen3.5 9B supports 262,144 tokens.