llmrun
Reasoning

ARC-AGI-2 Leaderboard

ARC-AGI-2 is the second-generation ARC Prize puzzle set, rebuilt so that tasks solved by brute-force search or memorization were removed and every problem still requires inferring a novel rule from a handful of examples. It is markedly harder than the original ARC-AGI, and most models — open and closed alike — score near zero.

Source: epoch16 open models ranked+174 proprietaryData through Sep 2026

Open modelsAll models

Open models ranked on ARC-AGI-2

# shows rank among open models / rank overall (including proprietary).

#ModelScore
1 / 49DeepSeek V4 Flash 0731 · 304.2B
61.4%
2 / 50DeepSeek V4 Pro 0813 · 1650.5B
61.3%
3 / 54Kimi K3 · 2779.9B
60.4%
4 / 68Inkling Small · 266.0B
40.1%
5 / 72Inkling · 952.4B
36.5%
6 / 83GLM 5.2 · 753.3B
22.8%
7 / 92Kimi K2.5 · 1026.9B
11.8%
8 / 114GLM 5 · 753.9B
4.9%
9 / 115MiniMax M2.5 · 228.7B
4.9%
10 / 121DeepSeek V3.2 · 685.4B
4.0%
11 / 151DeepSeek R1 · 684.5B
1.3%
12 / 158Qwen3 235B A22B Instruct 2507 · 235.1B
1.3%
13 / 159DeepSeek R1 0528 · 684.5B
1.1%
14 / 186Llama 4 Maverick 17B 128E Instruct · 401.6B
0.0%
15 / 187Llama 4 Scout 17B 16E Instruct · 108.6B
0.0%
16 / 189Magistral Small 2506 · 23.6B
0.0%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →61.4%0.0%DeepSeek V4 Pro 0813 · 1.7T · 61.3%Kimi K3 · 2.8T · 60.4%Inkling · 952B · 36.5%GLM 5.2 · 753B · 22.8%Kimi K2.5 · 1T · 11.8%GLM 5 · 754B · 4.9%DeepSeek V3.2 · 685B · 4.0%DeepSeek R1 · 684B · 1.3%Qwen3 235B A22B Instruct 2507 · 235B · 1.3%DeepSeek R1 0528 · 685B · 1.1%Llama 4 Maverick 17B 128E Instruct · 402B · 0.0%Llama 4 Scout 17B 16E Instruct · 109B · 0.0%Magistral Small 2506 · 24B · 0.0%Magistral Small 2506MiniMax M2.5 · 229B · 4.9%MiniMax M2.5Inkling Small · 266B · 40.1%Inkling SmallDeepSeek V4 Flash 0731 · 304B · 61.4%DeepSeek V4 Flash 0731
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Magistral Small 2506, 24B, score 0.0% — on the efficiency frontier (best score at its size or smaller).
  • MiniMax M2.5, 229B, score 4.9% — on the efficiency frontier (best score at its size or smaller).
  • Inkling Small, 266B, score 40.1% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Flash 0731, 304B, score 61.4% — on the efficiency frontier (best score at its size or smaller).

ARC-AGI-2: frequently asked questions

What is the best open LLM on ARC-AGI-2?
DeepSeek V4 Flash 0731 is the top open model on ARC-AGI-2, scoring 61.4%. Among all models tested — including proprietary ones — it ranks #49. The top model overall is GPT 6 Astra Max (OpenAI) at 95.0%.
What's the best ARC-AGI-2 model you can run on a 24 GB GPU?
Magistral Small 2506 is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 13 GB), scoring 0.0% on ARC-AGI-2.
Can open models match proprietary models on ARC-AGI-2?
Not quite on ARC-AGI-2: the strongest proprietary model (GPT 6 Astra Max) scores 95.0%, ahead of the best open model (DeepSeek V4 Flash 0731) at 61.4% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.