GSO-Bench Leaderboard
GSO-Bench, built by a team of UC Berkeley-affiliated researchers, gives a coding agent a real codebase and a performance test and asks it to optimize runtime efficiency to match an expert developer's speedup without breaking correctness, across 102 tasks in 10 codebases and five languages. The scored metric, Opt@1, is the share of tasks where a single attempt reaches at least 95% of the human speedup while still passing all tests.
Source: epoch3 open models ranked+39 proprietaryData through Sep 2026
All models ranked on GSO-Bench
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 | Claude Fable 5.1 (unspecified) · proprietary | 88.2% | API only |
| 2 | GPT 6 Astra (unspecified) · proprietary | 79.4% | API only |
| 3 | Claude Fable 5 (unspecified) · proprietary | 78.4% | API only |
| 4 | GPT 5.6 Sol (unspecified) · proprietary | 76.5% | API only |
| 5 | Claude Opus 4.8 (unspecified) · proprietary | 47.1% | API only |
| 6 | Claude Opus 4.7 · proprietary | 44.1% | API only |
| 7 | Claude Opus 4.7 (high) · proprietary | 44.1% | API only |
| 8 | Claude Opus 4.6 (high) · proprietary | 41.2% | API only |
| 9 | GPT 5.5 (xhigh) · proprietary | 40.2% | API only |
| 10 | Claude Sonnet 5 (unspecified) · proprietary | 37.3% | API only |
| 11 | Claude Opus 4.6 · proprietary | 33.3% | API only |
| 12 | Claude Opus 4.6 (unspecified) · proprietary | 33.3% | API only |
| 13 | GPT 5.4 (Mar 05, 2026, xhigh) · proprietary | 31.4% | API only |
| 14 | GPT 5.2 (Dec 11, 2025, high) · proprietary | 27.5% | API only |
| 15 | Claude Opus 4.5 (Nov 01, 2025) · proprietary | 26.5% | API only |
| 16 | Claude Opus 4.5 (Nov 01, 2025, unspecified) · proprietary | 26.5% | API only |
| 17 | GPT 5.4 (Mar 05, 2026, high) · proprietary | 25.5% | API only |
| 18 | Gemini 3.1 Pro Preview · proprietary | 22.6% | API only |
| 19 | Gemini 3 Pro Preview · proprietary | 18.6% | API only |
| 20 | Claude Sonnet 4.5 (Sep 29, 2025, unspecified) · proprietary | 14.7% | API only |
| 21 | Claude Sonnet 4.5 (Sep 29, 2025) · proprietary | 14.7% | API only |
| 22 | GPT 5.1 (Nov 13, 2025, unspecified) · proprietary | 13.7% | API only |
| 23 | GPT 5.1 (Nov 13, 2025, high) · proprietary | 13.7% | API only |
| 24 | Gemini 3 Flash Preview · proprietary | 9.8% | API only |
| 25 | O3 (Apr 16, 2025, high) · proprietary | 8.8% | API only |
| 26 | Claude Opus 4 (May 14, 2025) · proprietary | 6.9% | API only |
| 27 | Claude Opus 4 (May 14, 2025, unspecified) · proprietary | 6.9% | API only |
| 28 | GPT 5 (Aug 07, 2025, high) · proprietary | 6.9% | API only |
| 29 | Claude Sonnet 4 (May 14, 2025, unspecified) · proprietary | 4.9% | API only |
| 30 | Claude Sonnet 4 (May 14, 2025) · proprietary | 4.9% | API only |
| 31 | Kimi K2 Instruct · 1026.4B | 4.9% | ~620 GB |
| 32 | Qwen3 Coder 480B A35B Instruct · 480.2B | 4.9% | ~289 GB |
| 33 | Claude 3.5 Sonnet (Oct 22, 2024) · proprietary | 4.6% | API only |
| 34 | Gemini 2.5 Pro · proprietary | 3.9% | API only |
| 35 | Gemini 2.5 Pro Preview (Jun 05) · proprietary | 3.9% | API only |
| 36 | Claude 3.7 Sonnet (Feb 19, 2025, unspecified) · proprietary | 3.8% | API only |
| 37 | Claude 3.7 Sonnet (Feb 19, 2025) · proprietary | 3.8% | API only |
| 38 | O4 Mini (Apr 16, 2025, high) · proprietary | 3.6% | API only |
| 39 | GLM 4.5 Air · 110.5B | 2.9% | ~67 GB |
| 40 | O3 Mini (Jan 31, 2025, high) · proprietary | 1.3% | API only |
| 41 | O3 Mini (Jan 31, 2025, low) · proprietary | 1.3% | API only |
| 42 | GPT 4o (Nov 20, 2024) · proprietary | 0.0% | API only |
GSO-Bench: frequently asked questions
- What is the best open LLM on GSO-Bench?
- Kimi K2 Instruct is the top open model on GSO-Bench, scoring 4.9%. Among all models tested — including proprietary ones — it ranks #29. The top model overall is Claude Fable 5.1 (unspecified) (Anthropic) at 88.2%.
- Can open models match proprietary models on GSO-Bench?
- Not quite on GSO-Bench: the strongest proprietary model (Claude Fable 5.1 (unspecified)) scores 88.2%, ahead of the best open model (Kimi K2 Instruct) at 4.9% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.