Mistral 7B v0.1 — Benchmarks
Benchmark scores for Mistral 7B v0.1 aggregated from public leaderboards, with how it ranks among open models. See hardware requirements for what you need to run it.
Overall rank: #28 of 123 open modelscomposite 59.6/100 across 11 benchmarks in 3 categories · methodology
Knowledge
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| MMLU | 62.5% | #40 / 77 | #86 / 136 |
| HellaSwag | 81.0% | #13 / 42 | #26 / 76 |
| ARC Challenge | 78.6% | #12 / 51 | #18 / 77 |
| BoolQ | 87.4% | #5 / 33 | #15 / 77 |
| OpenBookQA | 79.8% | #5 / 21 | #7 / 42 |
| TriviaQA | 75.2% | #8 / 18 | #22 / 39 |
| MMLU-Pro | 30.9% | #95 / 119 | #225 / 259 |
Math
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| GSM8K | 54.4% | #27 / 60 | #40 / 93 |
Reasoning
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| BIG-Bench Hard | 56.1% | #13 / 37 | #20 / 50 |
| WinoGrande | 75.3% | #17 / 46 | #36 / 80 |
| PIQA | 83.0% | #8 / 35 | #14 / 60 |
Scores aggregated from public benchmark sources (each linked from the benchmark pages). llmrun does not run these benchmarks.