Gemma 7B — Benchmarks
Benchmark scores for Gemma 7B aggregated from public leaderboards, with how it ranks among open models. See hardware requirements for what you need to run it.
Overall rank: #33 of 123 open modelscomposite 57.3/100 across 11 benchmarks in 3 categories · methodology
Knowledge
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| MMLU | 66.1% | #34 / 77 | #77 / 136 |
| HellaSwag | 82.2% | #12 / 42 | #23 / 76 |
| ARC Challenge | 78.3% | #13 / 51 | #19 / 77 |
| BoolQ | 83.2% | #9 / 33 | #30 / 77 |
| OpenBookQA | 78.6% | #6 / 21 | #8 / 42 |
| TriviaQA | 72.3% | #10 / 18 | #27 / 39 |
| MMLU-Pro | 33.7% | #92 / 119 | #218 / 259 |
Math
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| GSM8K | 46.4% | #32 / 60 | #45 / 93 |
Reasoning
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| BIG-Bench Hard | 55.1% | #14 / 37 | #21 / 50 |
| WinoGrande | 79.0% | #11 / 46 | #22 / 80 |
| PIQA | 81.2% | #14 / 35 | #29 / 60 |
Scores aggregated from public benchmark sources (each linked from the benchmark pages). llmrun does not run these benchmarks.