Llama 3.1 70B Instruct vs Mixtral 8x22B Instruct v0.1
Llama 3.1 70B Instruct leads llmrun Score 77.9 to 71.6 (ranges overlap); neither fits 24 GB.
Model A · Meta
Llama 3.1 70B Instruct70.6B · 128K context · Llama 3.1 Community · Jul 2024
Model B · Mistral AI
Mixtral 8x22B Instruct v0.1140.6B · 39.2B active · 64K context · Apache 2.0 · Apr 2024
Choose Llama 3.1 70B Instruct if you want…
Higher llmrun Score
77.9 vs 71.6, ranges overlap
Stronger at reasoning
83.2 vs 76.2
Ahead on the benchmarks
5 of 5 shared boards
Fits a 48 GB GPU
Q4_K_M · 51.3 GB; Mixtral 8x22B Instruct v0.1 needs 42.7 GB at its smallest quant
Fits a Mac 64 GB
Q4_K_M · 51.3 GB; Mixtral 8x22B Instruct v0.1 needs 42.7 GB at its smallest quant
Lighter on memory
46.6 vs 85.2 GB at Q4_K_M
Choose Mixtral 8x22B Instruct v0.1 if you want…
More permissive license
Apache 2.0 vs Llama 3.1 Community
On your hardware
Each model at its default quant (Q4_K_M), a smaller one when it must. Estimates, ±20%.
8 GB
RTX 4060 · 8 GB tier
Llama 3.1 70B Instruct
Doesn't fit
smallest quant needs 26.1 GB
Mixtral 8x22B Instruct v0.1
Doesn't fit
smallest quant needs 42.7 GB
12 GB
RTX 3060 12GB · 12 GB tier
Llama 3.1 70B Instruct
Doesn't fit
smallest quant needs 26.1 GB
Mixtral 8x22B Instruct v0.1
Doesn't fit
smallest quant needs 42.7 GB
16 GB
RTX 5060 Ti 16GB · 16 GB tier
Llama 3.1 70B Instruct
Doesn't fit
smallest quant needs 26.1 GB
Mixtral 8x22B Instruct v0.1
Doesn't fit
smallest quant needs 42.7 GB
24 GBreference
RTX 4090 · 24 GB tier
Llama 3.1 70B Instruct
Doesn't fit
smallest quant needs 26.1 GB
Mixtral 8x22B Instruct v0.1
Doesn't fit
smallest quant needs 42.7 GB
32 GB
RTX 5090 · 32 GB tier
Llama 3.1 70B Instruct
IQ2_M · ~45 tok/s
30.9 GB · up to 16Kfits
Mixtral 8x22B Instruct v0.1
Doesn't fit
smallest quant needs 42.7 GB
48 GB
RTX A6000 · 48 GB tier
Llama 3.1 70B Instruct
Q4_K_M · ~11 tok/stight
51.3 GB · up to 4K
Mixtral 8x22B Instruct v0.1
IQ2_XS · ~37 tok/s
46.3 GB · up to 16Kmore headroom
Mac 32 GB
Mac Mini M4 · Macs
Llama 3.1 70B Instruct
IQ2_XS · ~4 tok/s
28 GB · up to 16Kfits
Mixtral 8x22B Instruct v0.1
Doesn't fit
smallest quant needs 42.7 GB
Mac 64 GB
Mac Studio M4 Max · Macs
Llama 3.1 70B Instruct
Q4_K_M · ~8 tok/s
51.3 GB · up to 32Kmore headroom
Mixtral 8x22B Instruct v0.1
IQ2_M · ~22 tok/s
51.5 GB · up to 32K
Mac 128 GB
Mac Studio M4 Max · Macs
Llama 3.1 70B Instruct
Q4_K_M · ~8 tok/s
51.3 GB · up to 128K
Mixtral 8x22B Instruct v0.1
Q4_K_M · ~14 tok/s
88.4 GB · up to 64Kfaster
Reference GPU: RTX 4090. Tinted cell = the better pick on that machine (fits, more headroom, or at least 10% faster).
Overall
Llama 3.1 70B Instruct
Mixtral 8x22B Instruct v0.1
Reasoning
Llama 3.1 70B Instruct
Mixtral 8x22B Instruct v0.1
Llama 3.1 70B Instruct · #87 of 100 open models · #37 of 49 that fit ≤64 GB
Mixtral 8x22B Instruct v0.1 · #92 of 100 open models · #53 of 60 that fit ≤96 GB
Benchmarks
5 shared boards: Llama 3.1 70B Instruct leads 5, Mixtral 8x22B Instruct v0.1 leads 0 · Benchmarks imported Oct 2026
Llama 3.1 70B Instruct
44.2%#66/98 open
IndependentEpoch AI · Jul 2024
Mixtral 8x22B Instruct v0.1
34.1%#76/98 open
IndependentEpoch AI · Apr 2024
Llama 3.1 70B Instruct
62.8%#53/145 open
IndependentTIGER-Lab
Mixtral 8x22B Instruct v0.1
56.3%#63/145 open
IndependentTIGER-Lab
Llama 3.1 70B Instruct
36.7%#23/39 open
IndependentEpoch AI · Jul 2024
Mixtral 8x22B Instruct v0.1
24.2%#27/39 open
IndependentEpoch AI · Apr 2024
Llama 3.1 70B Instruct
9.0%#44/46 open
IndependentEpoch AI · Jul 2024
Mixtral 8x22B Instruct v0.1
3.2%#45/46 open
IndependentEpoch AI · Apr 2024
Llama 3.1 70B Instruct
60.0%#48/68 open
Lab-reportedEpoch AI · Jul 2024
Mixtral 8x22B Instruct v0.1
55.1%#54/68 open
Lab-reportedEpoch AI · Apr 2024
6 more boards (other shared and single-model boards)
Llama 3.1 70B Instruct
1293.4#109/187 open
ArenaLMArena · Oct 2026
Mixtral 8x22B Instruct v0.1
1229.7#135/187 open
ArenaLMArena · Oct 2026
Only Llama 3.1 70B Instruct (4)
- AIME 2024/20253.6%
- MMLU80.1%
- BALROG27.9%
- LMCA14.8%
Only Mixtral 8x22B Instruct v0.1 (1)
- ForecastBench56.3
VRAM by quantization
Memory to load the model and 2K context; below it, the total at 32K and 128K context. Lighter side in bold.
Q2_K
Llama 3.1 70B Instruct
33.0 GB26.4 GB file
32K 43.1 GB · 128K 75.5 GB
Mixtral 8x22B Instruct v0.1
60.5 GB52.1 GB file
32K 67.6 GB · 128K —
Q3_K_S
Llama 3.1 70B Instruct
34.0 GB30.9 GB file
32K 44.1 GB · 128K 76.4 GB
Mixtral 8x22B Instruct v0.1
62.3 GB61.5 GB file
32K 69.4 GB · 128K —
Q3_K_M
Llama 3.1 70B Instruct
37.8 GB34.3 GB file
32K 48.0 GB · 128K 80.3 GB
Mixtral 8x22B Instruct v0.1
69.3 GB67.8 GB file
32K 76.4 GB · 128K —
Q4_0
Llama 3.1 70B Instruct
38.8 GB40.1 GB file
32K 48.9 GB · 128K 81.3 GB
Mixtral 8x22B Instruct v0.1
not listed
Q4_K_M
Llama 3.1 70B Instruct
46.6 GB42.5 GB file
32K 56.7 GB · 128K 89.1 GB
Mixtral 8x22B Instruct v0.1
85.2 GB85.6 GB file
32K 92.2 GB · 128K —
Q5_K_M
Llama 3.1 70B Instruct
55.3 GB49.9 GB file
32K 65.4 GB · 128K 97.8 GB
Mixtral 8x22B Instruct v0.1
101 GB100.0 GB file
32K 108 GB · 128K —
Q6_K
Llama 3.1 70B Instruct
64.0 GB57.9 GB file
32K 74.2 GB · 128K 107 GB
Mixtral 8x22B Instruct v0.1
117 GB~116.0 GB file
32K 124 GB · 128K —
Q8_0
Llama 3.1 70B Instruct
77.6 GB75.0 GB file
32K 87.7 GB · 128K 120 GB
Mixtral 8x22B Instruct v0.1
141 GB149.4 GB file
32K 148 GB · 128K —
16 more quants
IQ2_XXS
Llama 3.1 70B Instruct
21.3 GB19.1 GB file
32K 31.5 GB · 128K 63.8 GB
Mixtral 8x22B Instruct v0.1
39.4 GB37.9 GB file
32K 46.5 GB · 128K —
IQ2_XS
Llama 3.1 70B Instruct
23.3 GB21.1 GB file
32K 33.4 GB · 128K 65.8 GB
Mixtral 8x22B Instruct v0.1
43.0 GB42.0 GB file
32K 50.0 GB · 128K —
IQ2_S
Llama 3.1 70B Instruct
24.3 GB22.2 GB file
32K 34.4 GB · 128K 66.7 GB
Mixtral 8x22B Instruct v0.1
44.7 GB42.6 GB file
32K 51.8 GB · 128K —
IQ2_M
Llama 3.1 70B Instruct
26.2 GB24.1 GB file
32K 36.3 GB · 128K 68.7 GB
Mixtral 8x22B Instruct v0.1
48.2 GB46.7 GB file
32K 55.3 GB · 128K —
IQ3_XXS
Llama 3.1 70B Instruct
30.1 GB27.5 GB file
32K 40.2 GB · 128K 72.6 GB
Mixtral 8x22B Instruct v0.1
not listed
IQ3_XS
Llama 3.1 70B Instruct
32.0 GB29.3 GB file
32K 42.1 GB · 128K 74.5 GB
Mixtral 8x22B Instruct v0.1
58.8 GB58.2 GB file
32K 65.8 GB · 128K —
IQ3_S
Llama 3.1 70B Instruct
33.0 GB30.9 GB file
32K 43.1 GB · 128K 75.5 GB
Mixtral 8x22B Instruct v0.1
not listed
IQ3_M
Llama 3.1 70B Instruct
34.9 GB31.9 GB file
32K 45.0 GB · 128K 77.4 GB
Mixtral 8x22B Instruct v0.1
not listed
Q3_K_L
Llama 3.1 70B Instruct
39.8 GB37.1 GB file
32K 49.9 GB · 128K 82.3 GB
Mixtral 8x22B Instruct v0.1
72.8 GB72.6 GB file
32K 79.9 GB · 128K —
IQ4_XS
Llama 3.1 70B Instruct
41.7 GB37.9 GB file
32K 51.8 GB · 128K 84.2 GB
Mixtral 8x22B Instruct v0.1
76.4 GB76.4 GB file
32K 83.4 GB · 128K —
Q4_K_S
Llama 3.1 70B Instruct
43.7 GB40.3 GB file
32K 53.8 GB · 128K 86.2 GB
Mixtral 8x22B Instruct v0.1
79.9 GB80.5 GB file
32K 86.9 GB · 128K —
Q4_K_L
Llama 3.1 70B Instruct
47.5 GB43.3 GB file
32K 57.7 GB · 128K 90.0 GB
Mixtral 8x22B Instruct v0.1
not listed
Q5_0
Llama 3.1 70B Instruct
48.5 GB48.7 GB file
32K 58.6 GB · 128K 91.0 GB
Mixtral 8x22B Instruct v0.1
not listed
Q5_K_S
Llama 3.1 70B Instruct
53.4 GB48.7 GB file
32K 63.5 GB · 128K 95.9 GB
Mixtral 8x22B Instruct v0.1
97.5 GB97.0 GB file
32K 105 GB · 128K —
Q5_K_L
Llama 3.1 70B Instruct
56.3 GB50.6 GB file
32K 66.4 GB · 128K 98.8 GB
Mixtral 8x22B Instruct v0.1
not listed
BF16
Llama 3.1 70B Instruct
155 GB~141.1 GB file
32K 165 GB · 128K 198 GB
Mixtral 8x22B Instruct v0.1
282 GB~281.3 GB file
32K 289 GB · 128K —
Llama 3.1 70B Instruct
Ollama · Q4_K_M
ollama run hf.co/MaziyarPanahi/Meta-Llama-3.1-70B-Instruct-GGUF:Q4_K_MMixtral 8x22B Instruct v0.1
Ollama · Q4_K_M
ollama run hf.co/MaziyarPanahi/Mixtral-8x22B-Instruct-v0.1-GGUF:Q4_K_M| Spec | Llama 3.1 70B Instruct | Mixtral 8x22B Instruct v0.1 |
|---|---|---|
| Publisher | Meta | Mistral AI |
| Parameters | 70.6B | 140.6B |
| Active parameters | — | 39B per token |
| Architecture | — | MixtralForCausalLM |
| Context | 128K tokens | 64K tokens |
| Modalities | Chat, Function-calling | Chat, Function-calling |
| License | Llama 3.1 Community | Apache 2.0 |
| Released | Jul 2024 | Apr 2024 |
Frequently Asked Questions
Which is better, Llama 3.1 70B Instruct or Mixtral 8x22B Instruct v0.1?
Llama 3.1 70B Instruct leads llmrun Score 77.9 to 71.6 (ranges overlap); neither fits 24 GB. Choose Llama 3.1 70B Instruct for higher llmrun Score and stronger at reasoning; choose Mixtral 8x22B Instruct v0.1 for more permissive license.
Which needs less VRAM, Llama 3.1 70B Instruct or Mixtral 8x22B Instruct v0.1?
At Q4_K_M, Llama 3.1 70B Instruct requires 46.6 GB while Mixtral 8x22B Instruct v0.1 needs 85.2 GB, making Llama 3.1 70B Instruct the easier model to run on your local setup.
Can I run Llama 3.1 70B Instruct or Mixtral 8x22B Instruct v0.1 on a 16 GB GPU?
Llama 3.1 70B Instruct does not fit (smallest quant needs 26.1 GB); Mixtral 8x22B Instruct v0.1 does not fit (smallest quant needs 42.7 GB).
Which has a longer context window, Llama 3.1 70B Instruct or Mixtral 8x22B Instruct v0.1?
Llama 3.1 70B Instruct supports 131,072 tokens and Mixtral 8x22B Instruct v0.1 supports 65,536 tokens.