Starcoder2 15B — Benchmarks
Benchmark scores for Starcoder2 15B aggregated from public leaderboards, with how it ranks among open models. See hardware requirements for what you need to run it.
Overall rank: #65 of 123 open modelscomposite 45.3/100 across 4 benchmarks in 3 categories · methodology
Knowledge
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| ARC Challenge | 47.2% | #33 / 51 | #51 / 77 |
| MMLU | 64.1% | #36 / 77 | #80 / 136 |
Math
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| GSM8K | 57.7% | #26 / 60 | #36 / 93 |
Reasoning
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| WinoGrande | 64.3% | #35 / 46 | #65 / 80 |
Scores aggregated from public benchmark sources (each linked from the benchmark pages). llmrun does not run these benchmarks.