llmrun
Reasoning

ARC-AGI Leaderboard

ARC-AGI tests fluid, abstract reasoning on small visual grid puzzles where each task follows a novel rule the model must infer from a few examples. It deliberately resists memorization and is one of the most-watched measures of general reasoning progress.

Source: epoch16 open models ranked+184 proprietaryData through Sep 2026

Open modelsAll models

Open models ranked on ARC-AGI

# shows rank among open models / rank overall (including proprietary).

#ModelScore
1 / 27Kimi K3 · 2779.9B
94.5%
2 / 47DeepSeek V4 Pro 0813 · 1650.5B
90.5%
3 / 51DeepSeek V4 Flash 0731 · 304.2B
89.0%
4 / 68Inkling Small · 266.0B
84.0%
5 / 72Inkling · 952.4B
79.5%
6 / 75GLM 5.2 · 753.3B
77.0%
7 / 92Kimi K2.5 · 1026.9B
65.3%
8 / 95MiniMax M2.5 · 228.7B
63.7%
9 / 103DeepSeek V3.2 · 685.4B
57.0%
10 / 117GLM 5 · 753.9B
44.7%
11 / 163DeepSeek R1 0528 · 684.5B
21.2%
12 / 174DeepSeek R1 · 684.5B
15.8%
13 / 184Qwen3 235B A22B Instruct 2507 · 235.1B
11.0%
14 / 193Magistral Small 2506 · 23.6B
5.0%
15 / 195Llama 4 Maverick 17B 128E Instruct · 401.6B
4.4%
16 / 199Llama 4 Scout 17B 16E Instruct · 108.6B
0.5%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →94.5%0.5%Inkling · 952B · 79.5%GLM 5.2 · 753B · 77.0%Kimi K2.5 · 1T · 65.3%DeepSeek V3.2 · 685B · 57.0%GLM 5 · 754B · 44.7%DeepSeek R1 0528 · 685B · 21.2%DeepSeek R1 · 685B · 15.8%Qwen3 235B A22B Instruct 2507 · 235B · 11.0%Llama 4 Maverick 17B 128E Instruct · 402B · 4.4%Llama 4 Scout 17B 16E Instruct · 109B · 0.5%Magistral Small 2506 · 24B · 5.0%Magistral Small 2506MiniMax M2.5 · 229B · 63.7%MiniMax M2.5Inkling Small · 266B · 84.0%DeepSeek V4 Flash 0731 · 304B · 89.0%DeepSeek V4 Flash 0731DeepSeek V4 Pro 0813 · 1.7T · 90.5%DeepSeek V4 Pro 0813Kimi K3 · 2.8T · 94.5%Kimi K3
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Magistral Small 2506, 24B, score 5.0% — on the efficiency frontier (best score at its size or smaller).
  • MiniMax M2.5, 229B, score 63.7% — on the efficiency frontier (best score at its size or smaller).
  • Inkling Small, 266B, score 84.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Flash 0731, 304B, score 89.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Pro 0813, 1.7T, score 90.5% — on the efficiency frontier (best score at its size or smaller).
  • Kimi K3, 2.8T, score 94.5% — on the efficiency frontier (best score at its size or smaller).

ARC-AGI: frequently asked questions

What is the best open LLM on ARC-AGI?
Kimi K3 is the top open model on ARC-AGI, scoring 94.5%. Among all models tested — including proprietary ones — it ranks #24. The top model overall is Claude Fable 5 (xhigh) (Anthropic) at 98.5%.
What's the best ARC-AGI model you can run on a 24 GB GPU?
Magistral Small 2506 is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 13 GB), scoring 5.0% on ARC-AGI.
Can open models match proprietary models on ARC-AGI?
Not quite on ARC-AGI: the strongest proprietary model (Claude Fable 5 (xhigh)) scores 98.5%, ahead of the best open model (Kimi K3) at 94.5% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.