INTELLECT 1 Instruct — Benchmarks
Benchmark scores for INTELLECT 1 Instruct aggregated from public leaderboards, with how it ranks among open models. See hardware requirements for what you need to run it.
No overall rank for INTELLECT 1 Instruct: an overall score is only published when a model has been measured on enough of the benchmarks current models are still submitted to. Its individual scores below stand on their own — see the methodology for how the overall score is built.
Knowledge
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| ARC Challenge | 54.5% | #28 / 56 | #40 / 76 |
| MMLU | 49.9% | #63 / 85 | #109 / 136 |
| HellaSwag | 71.4% | #29 / 47 | #49 / 75 |
Math
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| GSM8K | 38.6% | #36 / 62 | #52 / 93 |
Reasoning
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| WinoGrande | 65.8% | #32 / 51 | #59 / 79 |
| BIG-Bench Hard | 34.8% | #33 / 38 | #45 / 50 |
Scores aggregated from public benchmark sources (each linked from the benchmark pages). llmrun does not run these benchmarks.