GPT J 6B — Benchmarks
Benchmark scores for GPT J 6B aggregated from public leaderboards, with how it ranks among open models. See hardware requirements for what you need to run it.
Knowledge
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| HellaSwag | 66.2% | #34 / 42 | #61 / 76 |
| ARC Challenge | 36.3% | #43 / 51 | #67 / 77 |
| BoolQ | 65.4% | #27 / 33 | #62 / 77 |
| OpenBookQA | 38.2% | #17 / 21 | #35 / 42 |
| MMLU | 25.7% | #77 / 77 | #135 / 136 |
Reasoning
| Benchmark | Score | Open rank | All models |
|---|---|---|---|
| WinoGrande | 64.5% | #34 / 46 | #64 / 80 |
| PIQA | 75.4% | #30 / 35 | #53 / 60 |
Scores aggregated from public benchmark sources (each linked from the benchmark pages). llmrun does not run these benchmarks.