Llama 3.1 405B — Benchmarks
Benchmark scores for Llama 3.1 405B aggregated from public leaderboards, with how it ranks among open models. See hardware requirements for what you need to run it.
Knowledge
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| HellaSwag | 89.2% | #1 / 42 | #3 / 76 |
| MMLU-Pro | 61.6% | #51 / 119 | #136 / 259 |
| ARC Challenge | 95.3% | #2 / 51 | #2 / 77 |
| TriviaQA | 82.7% | #3 / 18 | #11 / 39 |
| MMLU | 84.4% | #7 / 77 | #14 / 136 |
Reasoning
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| BIG-Bench Hard | 82.9% | #2 / 37 | #4 / 50 |
| WinoGrande | 89.2% | #1 / 46 | #1 / 80 |
| PIQA | 85.9% | #2 / 35 | #4 / 60 |
Scores aggregated from public benchmark sources (each linked from the benchmark pages). llmrun does not run these benchmarks.