Gemma 3 12B IT vs Mistral Small 3.1 24B Instruct 2503
Mistral Small 3.1 24B Instruct 2503 leads llmrun Score 78.8 to 74.2 (ranges overlap); both fit 24 GB at Q4_K_M; Gemma 3 12B IT is 1.9× faster on RTX 4090 (~82 vs ~43 tok/s).
Model B · Mistral AI
Mistral Small 3.1 24B Instruct 250324.0B · 128K context · Apache 2.0 · Mar 2025
Choose Gemma 3 12B IT if you want…
Fits a 12 GB GPU
Q4_K_M · 10.9 GB; Mistral Small 3.1 24B Instruct 2503 needs 10.3 GB at its smallest quant
1.9× faster on RTX 4090
~82 vs ~43 tok/s
Lighter on memory
8.0 vs 15.1 GB at Q4_K_M
Also does vision
Mistral Small 3.1 24B Instruct 2503 doesn't list it
Choose Mistral Small 3.1 24B Instruct 2503 if you want…
Higher llmrun Score
78.8 vs 74.2, ranges overlap
Stronger at reasoning
81.5 vs 66.0
Stronger at science
81.1 vs 71.7
Longer context window
128K vs 32K tokens
Also does chat
Gemma 3 12B IT doesn't list it
More permissive license
Apache 2.0 vs Gemma Terms
On your hardware
Each model at its default quant (Q4_K_M), a smaller one when it must. Estimates, ±20%.
8 GB
RTX 4060 · 8 GB tier
Gemma 3 12B IT
IQ2_M · ~39 tok/s
7.4 GB · up to 16Kfits
Mistral Small 3.1 24B Instruct 2503
Doesn't fit
smallest quant needs 10.3 GB
12 GB
RTX 3060 12GB · 12 GB tier
Gemma 3 12B IT
Q4_K_M · ~29 tok/s
10.9 GB · up to 16Kmore headroom
Mistral Small 3.1 24B Instruct 2503
IQ2_M · ~27 tok/s
11.8 GB · up to 16K
16 GB
RTX 5060 Ti 16GB · 16 GB tier
Gemma 3 12B IT
Q4_K_M · ~36 tok/s
10.9 GB · up to 32Kmore headroom
Mistral Small 3.1 24B Instruct 2503
Q4_K_M · ~19 tok/stight
18.1 GB · up to 4K
24 GBreference
RTX 4090 · 24 GB tier
Gemma 3 12B IT
Q4_K_M · ~82 tok/s
10.9 GB · up to 32Kfaster
Mistral Small 3.1 24B Instruct 2503
Q4_K_M · ~43 tok/s
18.1 GB · up to 32K
32 GB
RTX 5090 · 32 GB tier
Gemma 3 12B IT
Q4_K_M · ~145 tok/s
10.9 GB · up to 32Kfaster
Mistral Small 3.1 24B Instruct 2503
Q4_K_M · ~77 tok/s
18.1 GB · up to 64K
48 GB
RTX A6000 · 48 GB tier
Gemma 3 12B IT
Q4_K_M · ~62 tok/s
10.9 GB · up to 32Kfaster
Mistral Small 3.1 24B Instruct 2503
Q4_K_M · ~33 tok/s
18.1 GB · up to 128K
Mac 32 GB
Mac Mini M4 · Macs
Gemma 3 12B IT
Q4_K_M · ~10 tok/s
10.9 GB · up to 32Kfaster
Mistral Small 3.1 24B Instruct 2503
Q4_K_M · ~6 tok/s
18.1 GB · up to 64K
Mac 64 GB
Mac Studio M4 Max · Macs
Gemma 3 12B IT
Q4_K_M · ~48 tok/s
10.9 GB · up to 32Kfaster
Mistral Small 3.1 24B Instruct 2503
Q4_K_M · ~25 tok/s
18.1 GB · up to 128K
Mac 128 GB
Mac Studio M4 Max · Macs
Gemma 3 12B IT
Q4_K_M · ~48 tok/s
10.9 GB · up to 32Kfaster
Mistral Small 3.1 24B Instruct 2503
Q4_K_M · ~25 tok/s
18.1 GB · up to 128K
Reference GPU: RTX 4090. Tinted cell = the better pick on that machine (fits, more headroom, or at least 10% faster).
Overall
Gemma 3 12B IT
Mistral Small 3.1 24B Instruct 2503
Science
Gemma 3 12B IT
Mistral Small 3.1 24B Instruct 2503
Reasoning
Gemma 3 12B IT
Mistral Small 3.1 24B Instruct 2503
Gemma 3 12B IT · #82 of 92 open models · #26 of 34 that fit 24 GB
Mistral Small 3.1 24B Instruct 2503 · #79 of 92 open models · #25 of 34 that fit 24 GB
Benchmarks
7 shared boards: Gemma 3 12B IT leads 2, Mistral Small 3.1 24B Instruct 2503 leads 4 · Benchmarks imported Oct 2026
Gemma 3 12B IT
39.5%#70/97 open
IndependentEpoch AI · Mar 2025
Mistral Small 3.1 24B Instruct 2503
47.5%#58/97 open
IndependentEpoch AI · Mar 2025
Gemma 3 12B IT
1341.8#82/186 open
ArenaLMArena · Oct 2026
Mistral Small 3.1 24B Instruct 2503
1302.8#106/186 open
ArenaLMArena · Oct 2026
Gemma 3 12B IT
16.7%#54/85 open
IndependentEpoch AI · Mar 2025
Mistral Small 3.1 24B Instruct 2503
5.8%#66/85 open
IndependentEpoch AI · Mar 2025
Gemma 3 12B IT
17.4%#62/65 open
Lab-reportedEpoch AI · Mar 2025
Mistral Small 3.1 24B Instruct 2503
26.5%#55/65 open
Lab-reportedEpoch AI · Mar 2025
Gemma 3 12B IT
0.0%#48/67 open
IndependentEpoch AI · Mar 2025
Mistral Small 3.1 24B Instruct 2503
1.0%#43/67 open
IndependentEpoch AI · Mar 2025
Gemma 3 12B IT
0.0%#48/65 open
Lab-reportedEpoch AI · Mar 2025
Mistral Small 3.1 24B Instruct 2503
0.0%#59/65 open
Lab-reportedEpoch AI · Mar 2025
Gemma 3 12B IT
48.8%#62/68 open
Lab-reportedEpoch AI · Mar 2025
Mistral Small 3.1 24B Instruct 2503
58.6%#52/68 open
Lab-reportedEpoch AI · Mar 2025
3 more boards (other shared and single-model boards)
Only Mistral Small 3.1 24B Instruct 2503 (1)
- MATH Level 546.8%
VRAM by quantization
Memory to load the model and 2K context; below it, the total at 32K and 128K context. Lighter side in bold.
Q2_K
Gemma 3 12B IT
5.7 GB4.8 GB file
32K 11.9 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
10.9 GB8.9 GB file
32K 17.2 GB · 128K 37.3 GB
Q3_K_S
Gemma 3 12B IT
5.9 GB5.5 GB file
32K 12.1 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
11.2 GB10.4 GB file
32K 17.5 GB · 128K 37.6 GB
Q3_K_M
Gemma 3 12B IT
6.5 GB6.0 GB file
32K 12.7 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
12.4 GB11.5 GB file
32K 18.7 GB · 128K 38.8 GB
Q4_0
Gemma 3 12B IT
6.7 GB6.9 GB file
32K 12.9 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
12.7 GB13.5 GB file
32K 19.0 GB · 128K 39.2 GB
Q4_K_M
Gemma 3 12B IT
8.0 GB7.3 GB file
32K 14.2 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
15.1 GB14.3 GB file
32K 21.4 GB · 128K 41.6 GB
Q5_K_M
Gemma 3 12B IT
9.6 GB8.4 GB file
32K 15.7 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
17.8 GB16.8 GB file
32K 24.1 GB · 128K 44.3 GB
Q6_K
Gemma 3 12B IT
11.1 GB9.7 GB file
32K 17.3 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
20.5 GB19.3 GB file
32K 26.8 GB · 128K 47.0 GB
Q8_0
Gemma 3 12B IT
13.4 GB12.5 GB file
32K 19.6 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
24.7 GB25.1 GB file
32K 31.0 GB · 128K 51.2 GB
19 more quants
IQ2_XXS
Gemma 3 12B IT
3.7 GB3.6 GB file
32K 9.9 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
7.3 GB6.8 GB file
32K 13.6 GB · 128K 33.7 GB
IQ2_XS
Gemma 3 12B IT
4.0 GB3.8 GB file
32K 10.2 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
7.9 GB7.2 GB file
32K 14.2 GB · 128K 34.3 GB
IQ2_S
Gemma 3 12B IT
4.2 GB4.0 GB file
32K 10.4 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
8.2 GB7.5 GB file
32K 14.5 GB · 128K 34.6 GB
IQ2_M
Gemma 3 12B IT
4.5 GB4.4 GB file
32K 10.7 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
8.8 GB8.2 GB file
32K 15.1 GB · 128K 35.2 GB
IQ3_XXS
Gemma 3 12B IT
5.2 GB4.8 GB file
32K 11.4 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
10.0 GB9.4 GB file
32K 16.3 GB · 128K 36.4 GB
Q2_K_S
Gemma 3 12B IT
5.4 GB4.4 GB file
32K 11.6 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
10.3 GB8.3 GB file
32K 16.6 GB · 128K 36.7 GB
IQ3_XS
Gemma 3 12B IT
5.5 GB5.2 GB file
32K 11.7 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
10.6 GB9.9 GB file
32K 16.9 GB · 128K 37.0 GB
IQ3_S
Gemma 3 12B IT
5.7 GB5.5 GB file
32K 11.9 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
10.9 GB10.4 GB file
32K 17.2 GB · 128K 37.3 GB
IQ3_M
Gemma 3 12B IT
6.0 GB5.7 GB file
32K 12.2 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
11.5 GB10.7 GB file
32K 17.8 GB · 128K 37.9 GB
Q3_K_L
Gemma 3 12B IT
6.9 GB6.5 GB file
32K 13.1 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
13.0 GB12.4 GB file
32K 19.3 GB · 128K 39.5 GB
IQ4_XS
Gemma 3 12B IT
7.2 GB6.6 GB file
32K 13.4 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
13.6 GB12.8 GB file
32K 19.9 GB · 128K 40.1 GB
IQ4_NL
Gemma 3 12B IT
7.5 GB6.9 GB file
32K 13.7 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
14.2 GB13.5 GB file
32K 20.5 GB · 128K 40.7 GB
Q4_1
Gemma 3 12B IT
7.5 GB7.6 GB file
32K 13.7 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
14.2 GB14.9 GB file
32K 20.5 GB · 128K 40.7 GB
Q4_K_S
Gemma 3 12B IT
7.5 GB6.9 GB file
32K 13.7 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
14.2 GB13.5 GB file
32K 20.5 GB · 128K 40.7 GB
Q4_K_L
Gemma 3 12B IT
8.2 GB7.5 GB file
32K 14.4 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
15.4 GB14.8 GB file
32K 21.7 GB · 128K 41.9 GB
Q5_0
Gemma 3 12B IT
8.4 GB8.2 GB file
32K 14.6 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
15.7 GB16.3 GB file
32K 22.0 GB · 128K 42.2 GB
Q5_K_S
Gemma 3 12B IT
9.2 GB8.2 GB file
32K 15.4 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
17.2 GB16.3 GB file
32K 23.5 GB · 128K 43.7 GB
Q5_K_L
Gemma 3 12B IT
9.7 GB8.7 GB file
32K 15.9 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
18.1 GB17.2 GB file
32K 24.4 GB · 128K 44.6 GB
BF16
Gemma 3 12B IT
26.8 GB23.5 GB file
32K 33.0 GB · 128K —
Mistral Small 3.1 24B Instruct 2503
48.7 GB47.2 GB file
32K 55.0 GB · 128K 75.2 GB
Gemma 3 12B IT
Ollama · Q4_K_M
ollama run hf.co/MaziyarPanahi/gemma-3-12b-it-GGUF:Q4_K_MMistral Small 3.1 24B Instruct 2503
Ollama · Q4_K_M
ollama run hf.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF:Q4_K_M| Spec | Gemma 3 12B IT | Mistral Small 3.1 24B Instruct 2503 |
|---|---|---|
| Publisher | Mistral AI | |
| Parameters | 12.2B | 24.0B |
| Architecture | — | Mistral3ForConditionalGeneration |
| Context | 32K tokens | 128K tokens |
| Modalities | Vision | Chat |
| License | Gemma Terms | Apache 2.0 |
| Released | Mar 2025 | Mar 2025 |
Frequently Asked Questions
Which is better, Gemma 3 12B IT or Mistral Small 3.1 24B Instruct 2503?
Mistral Small 3.1 24B Instruct 2503 leads llmrun Score 78.8 to 74.2 (ranges overlap); both fit 24 GB at Q4_K_M; Gemma 3 12B IT is 1.9× faster on RTX 4090 (~82 vs ~43 tok/s). Choose Gemma 3 12B IT for fits a 12 GB GPU and 1.9× faster on RTX 4090; choose Mistral Small 3.1 24B Instruct 2503 for higher llmrun Score and stronger at reasoning.
Which needs less VRAM, Gemma 3 12B IT or Mistral Small 3.1 24B Instruct 2503?
At Q4_K_M, Gemma 3 12B IT requires 8.0 GB while Mistral Small 3.1 24B Instruct 2503 needs 15.1 GB, making Gemma 3 12B IT the easier model to run on your local setup.
Which is faster on RTX 4090?
Estimated decode speed on RTX 4090 is about 82 tok/s for Gemma 3 12B IT (Q4_K_M) and 43 tok/s for Mistral Small 3.1 24B Instruct 2503 (Q4_K_M), so Gemma 3 12B IT is faster. Estimates, ±20%.
Can I run Gemma 3 12B IT or Mistral Small 3.1 24B Instruct 2503 on a 16 GB GPU?
Gemma 3 12B IT fits at Q4_K_M; Mistral Small 3.1 24B Instruct 2503 fits at Q4_K_M.
Which has a longer context window, Gemma 3 12B IT or Mistral Small 3.1 24B Instruct 2503?
Gemma 3 12B IT supports 32,768 tokens and Mistral Small 3.1 24B Instruct 2503 supports 131,072 tokens.