llmrun
Coding

ALE-Bench Leaderboard

ALE-Bench, from Sakana AI and AtCoder, tests long-horizon algorithmic optimisation using problems drawn from AtCoder Heuristic Contests, where there is no single correct answer and solutions are scored on quality. Results are reported as an AtCoder-style performance rating, where higher is better.

Source: epoch51 open models ranked+87 proprietaryData through Nov 2026

Open models ranked on ALE-Bench

# shows rank among open models / rank overall (including proprietary).

#ModelScoreVRAM @ Q4
1 / 14Kimi K3.1 Pro · 2780.1B
1932.8
~1674 GB
2 / 21DeepSeek V4.1 Flash 1017 · 763.2B
1622.0
~458 GB
3 / 23Muse Spark Open 816B A42B · 822.0B
1590.1
~543 GB
4 / 29Kimi K3 · 2779.9B
1524.5
~1674 GB
5 / 34DeepSeek V4 Pro 0813 · 1650.5B
1403.2
~991 GB
6 / 38GLM 5.3 · 753.3B
1317.4
~456 GB
7 / 40DeepSeek V4 Flash 0731 · 304.2B
1306.1
~183 GB
8 / 46MiMo V2.6 Flash RL · 309B
1244.2
~186 GB
9 / 48Mistral Large 4 1T A52B · 1000B
1213.2
~604 GB
10 / 56MiMo V2.6 Pro RL · 1020B
1158.1
~613 GB
11 / 59Beam 1.0 501B A23B · 501.3B
1097.5
~301 GB
12 / 60Kimi K2.6 · 1026.9B
1092.7
~620 GB
13 / 61DeepSeek V4.1 Flash · 763.2B
1092.3
~458 GB
14 / 64GLM 5.2 · 753.3B
1047.0
~456 GB
15 / 66DeepSeek V4 Pro · 1598.8B
1006.1
~960 GB
16 / 70Inkling · 952.4B
946.0
~572 GB
17 / 73Gemma 4 26B A4B IT · 25.8B
927.2
~16 GB
18 / 74Gemma 4 31B IT · 31.3B
925.5
~20 GB
19 / 77MiMo V2.5 Pro · 1023.2B
899.8
~614 GB
20 / 78GLM 5.1 · 753.9B
887.1
~457 GB
21 / 79Kimi K2.7 Code · 1026.9B
886.2
~620 GB
22 / 81Kimi K2.5 · 1026.9B
821.6
~620 GB
23 / 83DeepSeek R1 0528 · 684.5B
804.1
~415 GB
24 / 90GLM 5 · 753.9B
765.6
~457 GB
25 / 92Mistral Medium 3.5 128B · 127.7B
764.0
~78 GB
26 / 93DeepSeek V3.1 Terminus · 684.5B
745.2
~415 GB
27 / 94MiMo v2 Flash · 309.8B
738.0
~186 GB
28 / 97Step 3.7 Flash · 201.4B
694.1
~133 GB
29 / 98DeepSeek V4 Flash · 290.9B
678.2
~175 GB
30 / 104MiniMax M3 · 427.0B
640.0
~257 GB
31 / 105MiniMax M2.1 · 228.7B
623.8
~138 GB
32 / 106Qwen3.5 397B A17B · 403.4B
621.9
~242 GB
33 / 107MiniMax M2.5 · 228.7B
618.2
~138 GB
34 / 109MiniMax M2.7 · 228.7B
599.3
~138 GB
35 / 110Kimi K2 Thinking · 1026.4B
597.5
~620 GB
36 / 112GPT OSS 120B · 116.8B
575.6
~71 GB
37 / 113GPT OSS 20B · 20.9B
566.0
~13 GB
38 / 115MiMo V2.5 · 310.8B
514.0
~187 GB
39 / 116Mistral Small 4 119B 2603 · 119.4B
497.6
~73 GB
40 / 117Qwen3 Coder 480B A35B Instruct · 480.2B
461.4
~289 GB
41 / 119Ring 2.6 1T · 1025.4B
432.6
~2057 GB
42 / 120GLM 4.7 · 358.3B
399.5
~216 GB
43 / 123Qwen3.5 27B · 27.8B
349.4
~17 GB
44 / 124GLM 4.5 · 358.3B
344.8
~216 GB
45 / 125GLM 4.6 · 356.8B
340.8
~215 GB
46 / 128GLM 5.3 Flash · 321.3B
303.6
~195 GB
47 / 130Kimi K2 Instruct 0905 · 1026.4B
267.1
~620 GB
48 / 132Mistral Large 3 675B Instruct 2512 · 675B
264.7
~446 GB
49 / 134Qwen3.5 35B A3B · 36.0B
221.8
~22 GB
50 / 135NVIDIA Nemotron 3 Super 120B A12B BF16 · 123.6B
213.9
~75 GB
51 / 137Llama 4 Maverick 17B 128E Instruct · 401.6B
173.0
~265 GB

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →1932.8173.0Muse Spark Open 816B A42B · 822B · 1590.1Kimi K3 · 2.8T · 1524.5DeepSeek V4 Pro 0813 · 1.7T · 1403.2MiMo V2.6 Flash RL · 309B · 1244.2Mistral Large 4 1T A52B · 1T · 1213.2MiMo V2.6 Pro RL · 1T · 1158.1Beam 1.0 501B A23B · 501B · 1097.5Kimi K2.6 · 1T · 1092.7DeepSeek V4.1 Flash · 763B · 1092.3GLM 5.2 · 753B · 1047.0DeepSeek V4 Pro · 1.6T · 1006.1Inkling · 952B · 946.0Gemma 4 31B IT · 31B · 925.5MiMo V2.5 Pro · 1T · 899.8GLM 5.1 · 754B · 887.1Kimi K2.7 Code · 1T · 886.2Kimi K2.5 · 1T · 821.6DeepSeek R1 0528 · 685B · 804.1GLM 5 · 754B · 765.6Mistral Medium 3.5 128B · 128B · 764.0DeepSeek V3.1 Terminus · 685B · 745.2MiMo v2 Flash · 310B · 738.0Step 3.7 Flash · 201B · 694.1DeepSeek V4 Flash · 291B · 678.2MiniMax M3 · 427B · 640.0MiniMax M2.1 · 229B · 623.8Qwen3.5 397B A17B · 403B · 621.9MiniMax M2.5 · 229B · 618.2MiniMax M2.7 · 229B · 599.3Kimi K2 Thinking · 1T · 597.5GPT OSS 120B · 117B · 575.6MiMo V2.5 · 311B · 514.0Mistral Small 4 119B 2603 · 119B · 497.6Qwen3 Coder 480B A35B Instruct · 480B · 461.4Ring 2.6 1T · 1T · 432.6GLM 4.7 · 358B · 399.5Qwen3.5 27B · 28B · 349.4GLM 4.5 · 358B · 344.8GLM 4.6 · 357B · 340.8GLM 5.3 Flash · 321B · 303.6Kimi K2 Instruct 0905 · 1T · 267.1Mistral Large 3 675B Instruct 2512 · 675B · 264.7Qwen3.5 35B A3B · 36B · 221.8NVIDIA Nemotron 3 Super 120B A12B BF16 · 124B · 213.9Llama 4 Maverick 17B 128E Instruct · 402B · 173.0GPT OSS 20B · 21B · 566.0GPT OSS 20BGemma 4 26B A4B IT · 26B · 927.2Gemma 4 26B A4B ITDeepSeek V4 Flash 0731 · 304B · 1306.1DeepSeek V4 Flash 0731GLM 5.3 · 753B · 1317.4GLM 5.3DeepSeek V4.1 Flash 1017 · 763B · 1622.0DeepSeek V4.1 Flash 1…Kimi K3.1 Pro · 2.8T · 1932.8Kimi K3.1 Pro
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • GPT OSS 20B, 21B, score 566.0 — on the efficiency frontier (best score at its size or smaller).
  • Gemma 4 26B A4B IT, 26B, score 927.2 — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Flash 0731, 304B, score 1306.1 — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3, 753B, score 1317.4 — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4.1 Flash 1017, 763B, score 1622.0 — on the efficiency frontier (best score at its size or smaller).
  • Kimi K3.1 Pro, 2.8T, score 1932.8 — on the efficiency frontier (best score at its size or smaller).

ALE-Bench: frequently asked questions

What is the best open LLM on ALE-Bench?
Kimi K3.1 Pro is the top open model on ALE-Bench, scoring 1932.8. Among all models tested — including proprietary ones — it ranks #14. The top model overall is GPT 6 Astra Max (OpenAI) at 2951.3.
What's the best ALE-Bench model you can run on a 24 GB GPU?
Gemma 4 26B A4B IT is the highest-scoring open model that fits in 24 GB at 4-bit quantization (~16 GB), scoring 927.2 on ALE-Bench.
Can open models match proprietary models on ALE-Bench?
Not quite on ALE-Bench: the strongest proprietary model (GPT 6 Astra Max) scores 2951.3, ahead of the best open model (Kimi K3.1 Pro) at 1932.8 — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.