Gemma 2 9B IT vs Mistral Nemo Instruct 2407
Gemma 2 9B IT leads llmrun Score 68.0 to 65.4 (ranges overlap); both fit 24 GB at Q4_K_M; Gemma 2 9B IT is 1.3× faster on RTX 4090 (~107 vs ~81 tok/s).
Model B · Mistral AI
Mistral Nemo Instruct 240712.2B · 128K context · Apache 2.0 · Jul 2024
Choose Gemma 2 9B IT if you want…
Higher llmrun Score
68.0 vs 65.4, ranges overlap
Fits an 8 GB GPU
Q4_K_M · 7.3 GB; Mistral Nemo Instruct 2407 needs 7 GB at its smallest quant
1.3× faster on RTX 4090
~107 vs ~81 tok/s
Lighter on memory
6.1 vs 8.1 GB at Q4_K_M
Choose Mistral Nemo Instruct 2407 if you want…
Longer context window
128K vs 8K tokens
Also does function-calling
Gemma 2 9B IT doesn't list it
More permissive license
Apache 2.0 vs Gemma Terms
On your hardware
Each model at its default quant (Q4_K_M), a smaller one when it must. Estimates, ±20%.
8 GB
RTX 4060 · 8 GB tier
Gemma 2 9B IT
Q4_K_M · ~29 tok/s
7.3 GB · up to 8Kmore headroom
Mistral Nemo Instruct 2407
IQ2_M · ~37 tok/s
7.8 GB · up to 16K
12 GB
RTX 3060 12GB · 12 GB tier
Gemma 2 9B IT
Q4_K_M · ~38 tok/s
7.3 GB · up to 8Kfaster
Mistral Nemo Instruct 2407
Q4_K_M · ~29 tok/s
11 GB · up to 16K
16 GB
RTX 5060 Ti 16GB · 16 GB tier
Gemma 2 9B IT
Q4_K_M · ~48 tok/s
7.3 GB · up to 8Kfaster
Mistral Nemo Instruct 2407
Q4_K_M · ~36 tok/s
11 GB · up to 32K
24 GBreference
RTX 4090 · 24 GB tier
Gemma 2 9B IT
Q4_K_M · ~107 tok/s
7.3 GB · up to 8Kfaster
Mistral Nemo Instruct 2407
Q4_K_M · ~81 tok/s
11 GB · up to 64K
32 GB
RTX 5090 · 32 GB tier
Gemma 2 9B IT
Q4_K_M · ~191 tok/s
7.3 GB · up to 8Kfaster
Mistral Nemo Instruct 2407
Q4_K_M · ~144 tok/s
11 GB · up to 64K
48 GB
RTX A6000 · 48 GB tier
Gemma 2 9B IT
Q4_K_M · ~82 tok/s
7.3 GB · up to 8Kfaster
Mistral Nemo Instruct 2407
Q4_K_M · ~62 tok/s
11 GB · up to 128K
Mac 32 GB
Mac Mini M4 · Macs
Gemma 2 9B IT
Q4_K_M · ~14 tok/s
7.3 GB · up to 8Kfaster
Mistral Nemo Instruct 2407
Q4_K_M · ~10 tok/s
11 GB · up to 64K
Mac 64 GB
Mac Studio M4 Max · Macs
Gemma 2 9B IT
Q4_K_M · ~63 tok/s
7.3 GB · up to 8Kfaster
Mistral Nemo Instruct 2407
Q4_K_M · ~47 tok/s
11 GB · up to 128K
Mac 128 GB
Mac Studio M4 Max · Macs
Gemma 2 9B IT
Q4_K_M · ~63 tok/s
7.3 GB · up to 8Kfaster
Mistral Nemo Instruct 2407
Q4_K_M · ~47 tok/s
11 GB · up to 128K
Reference GPU: RTX 4090. Tinted cell = the better pick on that machine (fits, more headroom, or at least 10% faster).
Overall
Gemma 2 9B IT
Mistral Nemo Instruct 2407
Gemma 2 9B IT · #93 of 100 open models · #29 of 35 that fit 24 GB
Mistral Nemo Instruct 2407 · #94 of 100 open models · #30 of 35 that fit 24 GB
Benchmarks
3 shared boards: Gemma 2 9B IT leads 2, Mistral Nemo Instruct 2407 leads 1 · Benchmarks imported Oct 2026
Gemma 2 9B IT
27.5%#86/98 open
IndependentEpoch AI · Jun 2024
Mistral Nemo Instruct 2407
29.9%#82/98 open
IndependentEpoch AI · Jul 2024
Gemma 2 9B IT
52.1%#73/145 open
IndependentTIGER-Lab
Mistral Nemo Instruct 2407
44.8%#87/145 open
IndependentTIGER-Lab
Gemma 2 9B IT
21.0%#31/39 open
IndependentEpoch AI · Jun 2024
Mistral Nemo Instruct 2407
10.8%#33/39 open
IndependentEpoch AI · Jul 2024
5 more boards (other shared and single-model boards)
Only Gemma 2 9B IT (3)
- LMArena Text1267.0
- AIME 2024/20250.6%
- MMLU72.1%
VRAM by quantization
Memory to load the model and 2K context; below it, the total at 32K and 128K context. Lighter side in bold.
Q2_K
Gemma 2 9B IT
4.3 GB3.8 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
5.9 GB4.8 GB file
32K 12.2 GB · 128K 32.3 GB
Q3_K_S
Gemma 2 9B IT
4.5 GB4.3 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
6.1 GB5.5 GB file
32K 12.4 GB · 128K 32.5 GB
Q3_K_M
Gemma 2 9B IT
5.0 GB4.8 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
6.7 GB6.1 GB file
32K 13.0 GB · 128K 33.1 GB
Q4_0
Gemma 2 9B IT
5.1 GB5.4 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
6.8 GB7.1 GB file
32K 13.1 GB · 128K 33.3 GB
Q4_K_M
Gemma 2 9B IT
6.1 GB5.8 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
8.1 GB7.5 GB file
32K 14.4 GB · 128K 34.5 GB
Q5_K_M
Gemma 2 9B IT
7.2 GB6.6 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
9.5 GB8.7 GB file
32K 15.7 GB · 128K 35.9 GB
Q6_K
Gemma 2 9B IT
8.4 GB7.6 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
10.8 GB10.1 GB file
32K 17.1 GB · 128K 37.2 GB
Q8_0
Gemma 2 9B IT
10.2 GB9.8 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
13.0 GB13.0 GB file
32K 19.3 GB · 128K 39.4 GB
18 more quants
IQ2_XXS
Gemma 2 9B IT
2.8 GB2.8 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
4.1 GB3.6 GB file
32K 10.4 GB · 128K 30.5 GB
IQ2_XS
Gemma 2 9B IT
3.1 GB3.1 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
4.4 GB3.9 GB file
32K 10.7 GB · 128K 30.8 GB
IQ2_S
Gemma 2 9B IT
3.2 GB3.2 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
4.6 GB4.1 GB file
32K 10.8 GB · 128K 31.0 GB
IQ2_M
Gemma 2 9B IT
3.4 GB3.4 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
4.9 GB4.4 GB file
32K 11.1 GB · 128K 31.3 GB
IQ3_XXS
Gemma 2 9B IT
3.9 GB3.8 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
5.5 GB4.9 GB file
32K 11.8 GB · 128K 31.9 GB
IQ3_XS
Gemma 2 9B IT
4.2 GB4.1 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
5.8 GB5.3 GB file
32K 12.1 GB · 128K 32.2 GB
IQ3_S
Gemma 2 9B IT
4.3 GB4.3 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
5.9 GB5.6 GB file
32K 12.2 GB · 128K 32.3 GB
IQ3_M
Gemma 2 9B IT
4.6 GB4.5 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
6.2 GB5.7 GB file
32K 12.5 GB · 128K 32.7 GB
Q3_K_L
Gemma 2 9B IT
5.2 GB5.1 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
7.0 GB6.6 GB file
32K 13.3 GB · 128K 33.4 GB
IQ4_XS
Gemma 2 9B IT
5.5 GB5.2 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
7.3 GB6.7 GB file
32K 13.6 GB · 128K 33.7 GB
Q4_1
Gemma 2 9B IT
5.7 GB6.0 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
not listed
Q4_K_S
Gemma 2 9B IT
5.7 GB5.5 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
7.6 GB7.1 GB file
32K 13.9 GB · 128K 34.0 GB
Q4_K_L
Gemma 2 9B IT
6.2 GB6.0 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
8.2 GB8.0 GB file
32K 14.5 GB · 128K 34.6 GB
Q5_0
Gemma 2 9B IT
6.4 GB6.5 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
8.4 GB8.5 GB file
32K 14.7 GB · 128K 34.8 GB
Q5_1
Gemma 2 9B IT
7.0 GB7.0 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
not listed
Q5_K_S
Gemma 2 9B IT
7.0 GB6.5 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
9.1 GB8.5 GB file
32K 15.4 GB · 128K 35.6 GB
Q5_K_L
Gemma 2 9B IT
7.4 GB6.9 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
9.6 GB9.1 GB file
32K 15.9 GB · 128K 36.0 GB
BF16
Gemma 2 9B IT
20.3 GB~18.5 GB file
32K — · 128K —
Mistral Nemo Instruct 2407
25.2 GB~24.5 GB file
32K 31.5 GB · 128K 51.6 GB
Gemma 2 9B IT
Ollama · Q4_K_M
ollama run hf.co/bartowski/gemma-2-9b-it-GGUF:Q4_K_MMistral Nemo Instruct 2407
Ollama · Q4_K_M
ollama run hf.co/MaziyarPanahi/Mistral-Nemo-Instruct-2407-GGUF:Q4_K_M| Spec | Gemma 2 9B IT | Mistral Nemo Instruct 2407 |
|---|---|---|
| Publisher | Mistral AI | |
| Parameters | 9.2B | 12.2B |
| Architecture | — | MistralForCausalLM |
| Context | 8K tokens | 128K tokens |
| Modalities | Chat | Chat, Function-calling |
| License | Gemma Terms | Apache 2.0 |
| Released | Jun 2024 | Jul 2024 |
Frequently Asked Questions
Which is better, Gemma 2 9B IT or Mistral Nemo Instruct 2407?
Gemma 2 9B IT leads llmrun Score 68.0 to 65.4 (ranges overlap); both fit 24 GB at Q4_K_M; Gemma 2 9B IT is 1.3× faster on RTX 4090 (~107 vs ~81 tok/s). Choose Gemma 2 9B IT for higher llmrun Score and fits an 8 GB GPU; choose Mistral Nemo Instruct 2407 for longer context window and also does function-calling.
Which needs less VRAM, Gemma 2 9B IT or Mistral Nemo Instruct 2407?
At Q4_K_M, Gemma 2 9B IT requires 6.1 GB while Mistral Nemo Instruct 2407 needs 8.1 GB, making Gemma 2 9B IT the easier model to run on your local setup.
Which is faster on RTX 4090?
Estimated decode speed on RTX 4090 is about 107 tok/s for Gemma 2 9B IT (Q4_K_M) and 81 tok/s for Mistral Nemo Instruct 2407 (Q4_K_M), so Gemma 2 9B IT is faster. Estimates, ±20%.
Can I run Gemma 2 9B IT or Mistral Nemo Instruct 2407 on a 16 GB GPU?
Gemma 2 9B IT fits at Q4_K_M; Mistral Nemo Instruct 2407 fits at Q4_K_M.
Which has a longer context window, Gemma 2 9B IT or Mistral Nemo Instruct 2407?
Gemma 2 9B IT supports 8,192 tokens and Mistral Nemo Instruct 2407 supports 131,072 tokens.