Llama 2 70B Chat HF vs Mistral Large Instruct 2407
Mistral Large Instruct 2407 leads llmrun Score 81.1 to 50.4; neither fits 24 GB.
Model B · Mistral AI
Mistral Large Instruct 2407122.6B · Other · Jul 2024
Choose Mistral Large Instruct 2407 if you want…
Higher llmrun Score
81.1 vs 50.4
Stronger at reasoning
83.1 vs 45.3
Ahead on the benchmarks
5 of 5 shared boards
Fits a Mac 128 GB
Q4_K_M · 86.4 GB; Llama 2 70B Chat HF needs 156.4 GB at its smallest quant
Also does function-calling
Llama 2 70B Chat HF doesn't list it
Newer release
Jul 2024 vs Jul 2023
Llama 2 70B Chat HF: no clear edge over Mistral Large Instruct 2407 in the data we have.
On your hardware
Each model at its default quant (BF16 / Q4_K_M), a smaller one when it must. Estimates, ±20%.
8 GB
RTX 4060 · 8 GB tier
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
Doesn't fit
smallest quant needs 42.6 GB
12 GB
RTX 3060 12GB · 12 GB tier
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
Doesn't fit
smallest quant needs 42.6 GB
16 GB
RTX 5060 Ti 16GB · 16 GB tier
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
Doesn't fit
smallest quant needs 42.6 GB
24 GBreference
RTX 4090 · 24 GB tier
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
Doesn't fit
smallest quant needs 42.6 GB
32 GB
RTX 5090 · 32 GB tier
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
Doesn't fit
smallest quant needs 42.6 GB
48 GB
RTX A6000 · 48 GB tier
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
IQ2_S · ~12 tok/s
47.7 GB · up to 16Kfits
Mac 32 GB
Mac Mini M4 · Macs
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
Doesn't fit
smallest quant needs 42.6 GB
Mac 64 GB
Mac Studio M4 Max · Macs
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
Q2_K_S · ~7 tok/s
59.5 GB · up to 16Kfits
Mac 128 GB
Mac Studio M4 Max · Macs
Llama 2 70B Chat HF
Doesn't fit
smallest quant needs 156.4 GB
Mistral Large Instruct 2407
Q4_K_M · ~5 tok/s
86.4 GB · up to 32Kfits
Reference GPU: RTX 4090. Tinted cell = the better pick on that machine (fits, more headroom, or at least 10% faster).
Overall
Llama 2 70B Chat HF
Mistral Large Instruct 2407
Reasoning
Llama 2 70B Chat HF
Mistral Large Instruct 2407
Llama 2 70B Chat HF · #100 of 100 open models · #78 of 78 that fit ≤192 GB
Mistral Large Instruct 2407 · #81 of 100 open models · #42 of 60 that fit ≤96 GB
Benchmarks
5 shared boards: Llama 2 70B Chat HF leads 0, Mistral Large Instruct 2407 leads 5 · Benchmarks imported Oct 2026
Llama 2 70B Chat HF
26.3%#88/98 open
IndependentEpoch AI · Jul 2023
Mistral Large Instruct 2407
49.0%#56/98 open
IndependentEpoch AI · Jul 2024
Llama 2 70B Chat HF
0.0%#86/86 open
IndependentEpoch AI · Jul 2023
Mistral Large Instruct 2407
8.5%#59/86 open
IndependentEpoch AI · Jul 2024
Llama 2 70B Chat HF
3.3%#39/39 open
IndependentEpoch AI · Jul 2023
Mistral Large Instruct 2407
44.8%#17/39 open
IndependentEpoch AI · Jul 2024
Llama 2 70B Chat HF
41.6%#68/68 open
Lab-reportedEpoch AI · Jul 2023
Mistral Large Instruct 2407
61.2%#45/68 open
Lab-reportedEpoch AI · Jul 2024
Llama 2 70B Chat HF
51.4#26/26 open
Lab-reportedEpoch AI · Jul 2023
Mistral Large Instruct 2407
57.1#20/26 open
Lab-reportedEpoch AI · Jul 2024
6 more boards (other shared and single-model boards)
Llama 2 70B Chat HF
1170.7#161/187 open
ArenaLMArena · Oct 2026
Mistral Large Instruct 2407
1314.4#98/187 open
ArenaLMArena · Oct 2026
Llama 2 70B Chat HF
59.9%#47/85 open
Lab-reportedEpoch AI · Jul 2023
Mistral Large Instruct 2407
80.0%#11/85 open
Lab-reportedEpoch AI · Jul 2024
Only Llama 2 70B Chat HF (2)
- BIG-Bench Hard58.5%
- GSM8K58.7%
Only Mistral Large Instruct 2407 (2)
- MMLU-Pro65.9%
- SimpleBench22.5%
VRAM by quantization
Memory to load the model and 2K context; below it, the total at 32K and 128K context. Lighter side in bold.
Q2_K
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
57.3 GB45.2 GB file
32K 69.1 GB · 128K 107 GB
Q3_K_S
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
59.0 GB52.8 GB file
32K 70.8 GB · 128K 109 GB
Q3_K_M
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
65.8 GB59.1 GB file
32K 77.6 GB · 128K 115 GB
Q4_0
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
67.4 GB69.1 GB file
32K 79.3 GB · 128K 117 GB
Q4_K_M
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
80.9 GB73.2 GB file
32K 92.7 GB · 128K 131 GB
Q5_K_M
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
96.1 GB86.5 GB file
32K 108 GB · 128K 146 GB
Q6_K
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
111 GB100.6 GB file
32K 123 GB · 128K 161 GB
Q8_0
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
135 GB130.3 GB file
32K 147 GB · 128K 184 GB
BF16
Llama 2 70B Chat HF
152 GB~137.9 GB file
32K 162 GB · 128K 194 GB
Mistral Large Instruct 2407
270 GB~245.2 GB file
32K 282 GB · 128K 319 GB
13 more quants
IQ2_XXS
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
37.1 GB32.4 GB file
32K 48.9 GB · 128K 86.7 GB
IQ2_XS
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
40.5 GB36.1 GB file
32K 52.3 GB · 128K 90.1 GB
IQ2_S
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
42.2 GB38.4 GB file
32K 54.0 GB · 128K 91.8 GB
IQ2_M
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
45.5 GB41.6 GB file
32K 57.3 GB · 128K 95.1 GB
IQ3_XXS
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
52.3 GB47.0 GB file
32K 64.1 GB · 128K 102 GB
Q2_K_S
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
54.0 GB41.6 GB file
32K 65.8 GB · 128K 104 GB
IQ3_XS
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
55.6 GB50.1 GB file
32K 67.4 GB · 128K 105 GB
IQ3_M
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
60.7 GB55.3 GB file
32K 72.5 GB · 128K 110 GB
Q3_K_L
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
69.1 GB64.6 GB file
32K 80.9 GB · 128K 119 GB
IQ4_XS
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
72.5 GB65.4 GB file
32K 84.3 GB · 128K 122 GB
Q4_K_S
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
75.9 GB69.6 GB file
32K 87.7 GB · 128K 125 GB
Q5_0
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
84.3 GB84.4 GB file
32K 96.1 GB · 128K 134 GB
Q5_K_S
Llama 2 70B Chat HF
not listed
Mistral Large Instruct 2407
92.7 GB84.4 GB file
32K 105 GB · 128K 142 GB
Llama 2 70B Chat HF
No local recipe · BF16
No GGUF repo entry exists for this model yet.
All ways to run Llama 2 70B Chat HF →Mistral Large Instruct 2407
Ollama · Q4_K_M
ollama run hf.co/MaziyarPanahi/Mistral-Large-Instruct-2407-GGUF:Q4_K_M| Spec | Llama 2 70B Chat HF | Mistral Large Instruct 2407 |
|---|---|---|
| Publisher | Meta | Mistral AI |
| Parameters | 69.0B | 122.6B |
| Modalities | Chat | Chat, Function-calling |
| License | Llama 2 Community | Other |
| Released | Jul 2023 | Jul 2024 |
Frequently Asked Questions
Which is better, Llama 2 70B Chat HF or Mistral Large Instruct 2407?
Mistral Large Instruct 2407 leads llmrun Score 81.1 to 50.4; neither fits 24 GB. Choose Mistral Large Instruct 2407 for higher llmrun Score and stronger at reasoning.
Can I run Llama 2 70B Chat HF or Mistral Large Instruct 2407 on a 16 GB GPU?
Llama 2 70B Chat HF does not fit (smallest quant needs 156.4 GB); Mistral Large Instruct 2407 does not fit (smallest quant needs 42.6 GB).