Coding
ALE-Bench Leaderboard
ALE-Bench, from Sakana AI and AtCoder, tests long-horizon algorithmic optimisation using problems drawn from AtCoder Heuristic Contests, where there is no single correct answer and solutions are scored on quality. Results are reported as an AtCoder-style performance rating, where higher is better.
Source: epoch41 open models ranked+75 proprietaryData through Sep 2026
Open models ranked on ALE-Bench
# shows rank among open models / rank overall (including proprietary).
| # | Model | Score |
|---|---|---|
| 1 / 16 | Kimi K3 · 2779.9B | 1524.5 |
| 2 / 21 | DeepSeek V4 Pro 0813 · 1650.5B | 1403.2 |
| 3 / 25 | GLM 5.3 · 753.3B | 1317.4 |
| 4 / 27 | DeepSeek V4 Flash 0731 · 304.2B | 1306.1 |
| 5 / 42 | Kimi K2.6 · 1026.9B | 1092.7 |
| 6 / 43 | DeepSeek V4.1 Flash · 763.2B | 1092.3 |
| 7 / 45 | GLM 5.2 · 753.3B | 1047.0 |
| 8 / 47 | DeepSeek V4 Pro · 1598.8B | 1006.1 |
| 9 / 50 | Inkling · 952.4B | 946.0 |
| 10 / 53 | Gemma 4 26B A4B IT · 25.8B | 927.2 |
| 11 / 54 | Gemma 4 31B IT · 31.3B | 925.5 |
| 12 / 57 | MiMo V2.5 Pro · 1023.2B | 899.8 |
| 13 / 58 | GLM 5.1 · 753.9B | 887.1 |
| 14 / 59 | Kimi K2.7 Code · 1026.9B | 886.2 |
| 15 / 61 | Kimi K2.5 · 1026.9B | 821.6 |
| 16 / 63 | DeepSeek R1 0528 · 684.5B | 804.1 |
| 17 / 70 | GLM 5 · 753.9B | 765.6 |
| 18 / 73 | DeepSeek V3.1 Terminus · 684.5B | 745.2 |
| 19 / 74 | MiMo v2 Flash · 309.8B | 738.0 |
| 20 / 77 | Step 3.7 Flash · 201.4B | 694.1 |
| 21 / 78 | DeepSeek V4 Flash · 290.9B | 678.2 |
| 22 / 84 | MiniMax M3 · 427.0B | 640.0 |
| 23 / 85 | MiniMax M2.1 · 228.7B | 623.8 |
| 24 / 87 | MiniMax M2.5 · 228.7B | 618.2 |
| 25 / 88 | MiniMax M2.7 · 228.7B | 599.3 |
| 26 / 89 | Kimi K2 Thinking · 1026.4B | 597.5 |
| 27 / 91 | GPT OSS 120B · 116.8B | 575.6 |
| 28 / 92 | GPT OSS 20B · 20.9B | 566.0 |
| 29 / 94 | MiMo V2.5 · 310.8B | 514.0 |
| 30 / 95 | Mistral Small 4 119B 2603 · 119.4B | 497.6 |
| 31 / 96 | Qwen3 Coder 480B A35B Instruct · 480.2B | 461.4 |
| 32 / 98 | Ring 2.6 1T · 1025.4B | 432.6 |
| 33 / 99 | GLM 4.7 · 358.3B | 399.5 |
| 34 / 102 | Qwen3.5 27B · 27.8B | 349.4 |
| 35 / 103 | GLM 4.5 · 358.3B | 344.8 |
| 36 / 104 | GLM 4.6 · 356.8B | 340.8 |
| 37 / 107 | GLM 5.3 Flash · 321.3B | 303.6 |
| 38 / 109 | Kimi K2 Instruct 0905 · 1026.5B | 267.1 |
| 39 / 110 | Mistral Large 3 675B Instruct 2512 · 675B | 264.7 |
| 40 / 113 | NVIDIA Nemotron 3 Super 120B A12B BF16 · 123.6B | 213.9 |
| 41 / 115 | Llama 4 Maverick 17B 128E Instruct · 401.6B | 173.0 |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- GPT OSS 20B, 21B, score 566.0 — on the efficiency frontier (best score at its size or smaller).
- Gemma 4 26B A4B IT, 26B, score 927.2 — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Flash 0731, 304B, score 1306.1 — on the efficiency frontier (best score at its size or smaller).
- GLM 5.3, 753B, score 1317.4 — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Pro 0813, 1.7T, score 1403.2 — on the efficiency frontier (best score at its size or smaller).
- Kimi K3, 2.8T, score 1524.5 — on the efficiency frontier (best score at its size or smaller).
ALE-Bench: frequently asked questions
- What is the best open LLM on ALE-Bench?
- Kimi K3 is the top open model on ALE-Bench, scoring 1524.5. Among all models tested — including proprietary ones — it ranks #16. The top model overall is GPT 6 Astra Max (OpenAI) at 2951.3.
- What's the best ALE-Bench model you can run on a 24 GB GPU?
- Gemma 4 26B A4B IT is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 14 GB), scoring 927.2 on ALE-Bench.
- What's the best ALE-Bench model you can run on a 12 GB GPU?
- GPT OSS 20B is the highest-scoring open model that fits in 12 GB at 4-bit quantization (about 12 GB), scoring 566.0 on ALE-Bench.
- Can open models match proprietary models on ALE-Bench?
- Not quite on ALE-Bench: the strongest proprietary model (GPT 6 Astra Max) scores 2951.3, ahead of the best open model (Kimi K3) at 1524.5 — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.