llmrun
Reasoning

ARC-AGI Leaderboard

ARC-AGI tests fluid, abstract reasoning on small visual grid puzzles where each task follows a novel rule the model must infer from a few examples. It deliberately resists memorization and is one of the most-watched measures of general reasoning progress.

Source: epoch16 open models ranked+184 proprietaryData through Sep 2026

Open modelsAll models

All models ranked on ARC-AGI

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScore
1Claude Fable 5 (xhigh) · proprietary
98.5%
2Claude Fable 5 Max · proprietary
98.5%
3GPT 6 Astra (high) · proprietary
98.5%
4GPT 6 Astra (xhigh) · proprietary
98.5%
5Gemini 3.1 Pro Preview · proprietary
98.0%
6Claude Fable 5.1 Max · proprietary
97.5%
7Claude Opus 5 (high) · proprietary
97.5%
8Claude Opus 5 Max · proprietary
97.5%
9GPT 5.6 Sol (xhigh) · proprietary
97.5%
10GPT 6 Astra (medium) · proprietary
97.5%
11GPT 6 Astra Max · proprietary
97.5%
12GPT 5.6 Sol (high) · proprietary
97.0%
13Claude Fable 5.1 (xhigh) · proprietary
96.5%
14GPT 5.5 Pro (high) · proprietary
96.5%
15GPT 5.6 Sol Max · proprietary
96.5%
16GPT 5.6 Terra Max · proprietary
96.5%
17GPT 6 Astra (low) · proprietary
96.5%
18Claude Fable 5.1 (high) · proprietary
96.0%
19Gemini 3 Deep Think Preview · proprietary
96.0%
20Claude Fable 5 (high) · proprietary
95.5%
21Gemini 3.7 Flash (high) · proprietary
95.5%
22GPT 5.5 (xhigh) · proprietary
95.0%
23GPT 5.5 Pro (xhigh) · proprietary
95.0%
24Claude Fable 5.1 (medium) · proprietary
94.5%
25GPT 5.4 Pro (Mar 05, 2026, xhigh) · proprietary
94.5%
26GPT 5.5 (high) · proprietary
94.5%
27Kimi K3 · 2779.9B
94.5%
28Claude Opus 4.6 (120K) · proprietary
94.0%
29GPT 5.6 Terra (xhigh) · proprietary
94.0%
30GPT 5.4 (Mar 05, 2026, xhigh) · proprietary
93.7%
31Claude Opus 4.7 (high) · proprietary
93.5%
32GPT 5.4 (Mar 05, 2026, high) · proprietary
92.7%
33Claude Fable 5 (medium) · proprietary
92.5%
34Claude Opus 4.8 Max · proprietary
92.5%
35Gemini 3.5 Flash (high) · proprietary
92.5%
36GPT 5.6 Sol (medium) · proprietary
92.5%
37GPT 5.5 (medium) · proprietary
92.2%
38Claude Opus 4.7 Max · proprietary
92.0%
39Claude Opus 4.8 (high) · proprietary
92.0%
40GPT 5.6 Terra (high) · proprietary
92.0%
41Claude Opus 4.8 (medium) · proprietary
91.5%
42Gemini 3.6 Flash (high) · proprietary
91.2%
43Gemini 3.7 Flash (medium) · proprietary
91.2%
44Claude Opus 4.7 (low) · proprietary
91.0%
45Claude Opus 4.7 (medium) · proprietary
91.0%
46Claude Fable 5 (low) · proprietary
90.5%
47DeepSeek V4 Pro 0813 · 1650.5B
90.5%
48GPT 5.2 Pro (Dec 11, 2025, xhigh) · proprietary
90.5%
49Claude Fable 5.1 (low) · proprietary
90.0%
50Grok 4.20 · proprietary
89.5%
51DeepSeek V4 Flash 0731 · 304.2B
89.0%
52Claude Opus 4.8 (low) · proprietary
88.0%
53GPT 5.6 Luna Max · proprietary
88.0%
54GPT 5.6 Luna (xhigh) · proprietary
87.7%
55Grok 4.6 (medium) · proprietary
87.5%
56Grok 4.5 (medium) · proprietary
87.2%
57Grok 4.6 (high) · proprietary
87.0%
58Grok 4.6 (xhigh) · proprietary
87.0%
59Claude Sonnet 4.6 (high) · proprietary
86.5%
60GPT 5.2 (Dec 11, 2025, xhigh) · proprietary
86.2%
61GPT 5.4 (Mar 05, 2026, medium) · proprietary
86.2%
62Claude Sonnet 4.6 Max · proprietary
86.0%
63GPT 6 Astra None · proprietary
86.0%
64GPT 5.2 Pro (Dec 11, 2025, high) · proprietary
85.7%
65Grok 4.5 (high) · proprietary
85.7%
66Gemini 3.7 Flash (low) · proprietary
85.2%
67Gemini 3 Flash Preview · proprietary
84.7%
68Inkling Small · 266.0B
84.0%
69Gemini 3.6 Flash (medium) · proprietary
83.2%
70GPT 5.2 Pro (Dec 11, 2025, medium) · proprietary
81.2%
71Claude Opus 4.5 (Nov 01, 2025, 64K) · proprietary
80.0%
72Inkling · 952.4B
79.5%
73Grok 4.5 (low) · proprietary
79.2%
74GPT 5.2 (Dec 11, 2025, high) · proprietary
78.7%
75GLM 5.2 · 753.3B
77.0%
76GPT 5.6 Terra (medium) · proprietary
77.0%
77Gemini 3.6 Flash (low) · proprietary
76.5%
78GPT 5.6 Luna (high) · proprietary
76.5%
79GPT 5.5 (low) · proprietary
76.2%
80Claude Opus 4.5 (Nov 01, 2025, 32K) · proprietary
75.8%
81Gemini 3 Pro Preview · proprietary
75.0%
82Grok 4.6 (low) · proprietary
74.8%
83GPT 5.6 Sol (low) · proprietary
74.5%
84GPT 5.1 (Nov 13, 2025, high) · proprietary
72.8%
85GPT 5.2 (Dec 11, 2025, medium) · proprietary
72.7%
86Claude Opus 4.5 (Nov 01, 2025, 16K) · proprietary
72.0%
87GPT 5 Pro (Oct 06, 2025, high) · proprietary
70.2%
88GPT 5 Pro (Oct 06, 2025, unspecified) · proprietary
70.2%
89GPT 5.4 (Mar 05, 2026, low) · proprietary
68.2%
90Grok 4 (Jul 09) · proprietary
66.7%
91GPT 5 (Aug 07, 2025, high) · proprietary
65.7%
92Kimi K2.5 · 1026.9B
65.3%
93Claude Sonnet 4.5 (Sep 29, 2025, 32K) · proprietary
63.7%
94GPT 5.4 Mini (Mar 17, 2026, xhigh) · proprietary
63.7%
95MiniMax M2.5 · 228.7B
63.7%
96O3 (Apr 16, 2025, high) · proprietary
60.8%
97GPT 5.6 Terra (low) · proprietary
60.2%
98O3 Pro (Jun 10, 2025, high) · proprietary
59.3%
99Claude Opus 4.5 (Nov 01, 2025, 8K) · proprietary
58.7%
100O4 Mini (Apr 16, 2025, high) · proprietary
58.7%
101GPT 5.4 Mini (Mar 17, 2026, high) · proprietary
58.0%
102GPT 5.1 (Nov 13, 2025, medium) · proprietary
57.7%
103DeepSeek V3.2 · 685.4B
57.0%
104O3 Pro (Jun 10, 2025, medium) · proprietary
57.0%
105GPT 5.6 Luna (medium) · proprietary
56.5%
106GPT 5 (Aug 07, 2025, medium) · proprietary
56.2%
107GPT 5.2 (Dec 11, 2025, low) · proprietary
55.7%
108GPT 5 Mini (Aug 07, 2025, high) · proprietary
54.3%
109O3 (Apr 16, 2025, medium) · proprietary
53.8%
110Gemini 3.5 Flash Lite (high) · proprietary
53.5%
111GPT 5.4 Nano (Mar 17, 2026, xhigh) · proprietary
51.5%
112Gemini 3.5 Flash (minimal) · proprietary
48.8%
113Grok 4 Fast · proprietary
48.5%
114Claude Sonnet 4.5 (Sep 29, 2025, 16K) · proprietary
48.3%
115Claude Haiku 4.5 (Oct 01, 2025, 32K) · proprietary
47.7%
116Claude Sonnet 4.5 (Sep 29, 2025, 8K) · proprietary
46.5%
117GLM 5 · 753.9B
44.7%
118O3 Pro (Jun 10, 2025, low) · proprietary
44.3%
119GPT 5 (Aug 07, 2025, low) · proprietary
44.0%
120O4 Mini (Apr 16, 2025, medium) · proprietary
41.8%
121O3 (Apr 16, 2025, low) · proprietary
41.5%
122Gemini 2.5 Pro (16K) · proprietary
41.0%
123GPT 5.4 Mini (Mar 17, 2026, medium) · proprietary
40.8%
124Claude Opus 4.5 (Nov 01, 2025) · proprietary
40.0%
125Claude Sonnet 4 (May 14, 2025, 16K) · proprietary
40.0%
126Tiny Recursion Model · proprietary
40.0%
127GPT 5.4 Nano (Mar 17, 2026, high) · proprietary
38.2%
128Claude Haiku 4.5 (Oct 01, 2025, 16K) · proprietary
37.3%
129GPT 5 Mini (Aug 07, 2025, medium) · proprietary
37.3%
130Gemini 2.5 Pro (32K) · proprietary
37.0%
131Claude Opus 4 (May 14, 2025, 16K) · proprietary
35.7%
132Gemini 3.6 Flash (minimal) · proprietary
34.5%
133O3 Mini (Jan 31, 2025, high) · proprietary
34.5%
134GPT 5.6 Luna (low) · proprietary
34.2%
135Gemini 2.5 Flash Preview (May 20, 16K) · proprietary
33.3%
136Gemini 2.5 Flash Preview (May 20) · proprietary
33.3%
137GPT 5.1 (Nov 13, 2025, low) · proprietary
33.2%
138Gemini 2.5 Pro Preview (Mar 25) · proprietary
33.0%
139GPT 5.4 Nano (Mar 17, 2026, medium) · proprietary
33.0%
140Gemini 2.5 Flash Preview (May 20, 23K) · proprietary
32.3%
141Gemini 3.5 Flash Lite (medium) · proprietary
32.3%
142Gemini 2.5 Flash Preview (24K thinking) (Apr 17) · proprietary
32.3%
143Gemini 2.5 Pro Preview (Jun 05, 1K) · proprietary
31.3%
144Claude Sonnet 4.5 (Sep 29, 2025, 1K) · proprietary
31.0%
145O1 (Dec 17, 2024, medium) · proprietary
30.7%
146Claude Opus 4 (May 14, 2025, 8K) · proprietary
30.7%
147Gemini 2.5 Pro (8K) · proprietary
29.5%
148Claude Sonnet 4 (May 14, 2025, 8K) · proprietary
29.0%
149Claude 3.7 Sonnet (Feb 19, 2025, 16K) · proprietary
28.6%
150Claude Sonnet 4 (May 14, 2025, 1K) · proprietary
28.0%
151Codex Mini (May 16, 2025) · proprietary
27.3%
152O1 (Dec 17, 2024, low) · proprietary
27.2%
153Claude Opus 4 (May 14, 2025, 1K) · proprietary
27.0%
154GPT 5 Mini (Aug 07, 2025, low) · proprietary
26.3%
155Gemini 2.5 Flash Preview (May 20, 8K) · proprietary
25.8%
156Claude Haiku 4.5 (Oct 01, 2025, 8K) · proprietary
25.5%
157Claude Sonnet 4.5 (Sep 29, 2025) · proprietary
25.5%
158Claude Sonnet 4 (May 14, 2025) · proprietary
23.8%
159O1 Pro (Mar 19, 2025, low) · proprietary
23.3%
160Claude Opus 4 (May 14, 2025) · proprietary
22.5%
161O3 Mini (Jan 31, 2025, medium) · proprietary
22.3%
162O4 Mini (Apr 16, 2025, low) · proprietary
21.3%
163DeepSeek R1 0528 · 684.5B
21.2%
164Claude 3.7 Sonnet (Feb 19, 2025, 8K) · proprietary
21.2%
165GPT 5 Nano (Aug 07, 2025, medium) · proprietary
20.7%
166GPT 5.4 Nano (Mar 17, 2026, low) · proprietary
18.3%
167O1 Preview (Sep 12, 2024) · proprietary
18.0%
168Gemini 3.5 Flash Lite (low) · proprietary
17.0%
169Claude Haiku 4.5 (Oct 01, 2025, 1K) · proprietary
16.8%
170GPT 5 Nano (Aug 07, 2025, high) · proprietary
16.7%
171Grok 3 Mini (low) · proprietary
16.5%
172Grok 3 Mini Beta (low) · proprietary
16.5%
173Gemini 2.5 Flash Preview (May 20, 1K) · proprietary
16.0%
174DeepSeek R1 · 684.5B
15.8%
175O3 Mini (Jan 31, 2025, low) · proprietary
14.5%
176Claude Haiku 4.5 (Oct 01, 2025) · proprietary
14.3%
177O1 Mini (Sep 12, 2024, medium) · proprietary
14.0%
178O1 Mini (Sep 12, 2024, unspecified) · proprietary
14.0%
179Claude 3.7 Sonnet (Feb 19, 2025) · proprietary
13.6%
180GPT 5.4 Mini (Mar 17, 2026, low) · proprietary
13.0%
181GPT 5.2 (Dec 11, 2025, unspecified) · proprietary
12.3%
182GPT 5.2 2025 12.11 None · proprietary
12.3%
183Claude 3.7 Sonnet (Feb 19, 2025, 1K) · proprietary
11.6%
184Qwen3 235B A22B Instruct 2507 · 235.1B
11.0%
185GPT 4.5 Preview (Feb 27, 2025) · proprietary
10.3%
186Gemini 3.5 Flash Lite (minimal) · proprietary
7.5%
187Magistral Medium 2506 · proprietary
6.1%
188GPT 5 (Aug 07, 2025, minimal) · proprietary
6.0%
189GPT 5.1 2025 11.13 None · proprietary
5.8%
190GPT 4.1 (Apr 14, 2025) · proprietary
5.5%
191Grok 3 · proprietary
5.5%
192GPT 5 Mini (Aug 07, 2025, minimal) · proprietary
5.3%
193Magistral Small 2506 · 23.6B
5.0%
194GPT 4o (Nov 20, 2024) · proprietary
4.5%
195Llama 4 Maverick 17B 128E Instruct · 401.6B
4.4%
196GPT 5 Nano (Aug 07, 2025, low) · proprietary
4.0%
197GPT 4.1 Mini (Apr 14, 2025) · proprietary
3.5%
198GPT 5 Nano (Aug 07, 2025, minimal) · proprietary
1.5%
199Llama 4 Scout 17B 16E Instruct · 108.6B
0.5%
200GPT 4.1 Nano (Apr 14, 2025) · proprietary
0.0%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →94.5%0.5%Inkling · 952B · 79.5%GLM 5.2 · 753B · 77.0%Kimi K2.5 · 1T · 65.3%DeepSeek V3.2 · 685B · 57.0%GLM 5 · 754B · 44.7%DeepSeek R1 0528 · 685B · 21.2%DeepSeek R1 · 685B · 15.8%Qwen3 235B A22B Instruct 2507 · 235B · 11.0%Llama 4 Maverick 17B 128E Instruct · 402B · 4.4%Llama 4 Scout 17B 16E Instruct · 109B · 0.5%Magistral Small 2506 · 24B · 5.0%Magistral Small 2506MiniMax M2.5 · 229B · 63.7%MiniMax M2.5Inkling Small · 266B · 84.0%DeepSeek V4 Flash 0731 · 304B · 89.0%DeepSeek V4 Flash 0731DeepSeek V4 Pro 0813 · 1.7T · 90.5%DeepSeek V4 Pro 0813Kimi K3 · 2.8T · 94.5%Kimi K3
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Magistral Small 2506, 24B, score 5.0% — on the efficiency frontier (best score at its size or smaller).
  • MiniMax M2.5, 229B, score 63.7% — on the efficiency frontier (best score at its size or smaller).
  • Inkling Small, 266B, score 84.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Flash 0731, 304B, score 89.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Pro 0813, 1.7T, score 90.5% — on the efficiency frontier (best score at its size or smaller).
  • Kimi K3, 2.8T, score 94.5% — on the efficiency frontier (best score at its size or smaller).

ARC-AGI: frequently asked questions

What is the best open LLM on ARC-AGI?
Kimi K3 is the top open model on ARC-AGI, scoring 94.5%. Among all models tested — including proprietary ones — it ranks #24. The top model overall is Claude Fable 5 (xhigh) (Anthropic) at 98.5%.
What's the best ARC-AGI model you can run on a 24 GB GPU?
Magistral Small 2506 is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 13 GB), scoring 5.0% on ARC-AGI.
Can open models match proprietary models on ARC-AGI?
Not quite on ARC-AGI: the strongest proprietary model (Claude Fable 5 (xhigh)) scores 98.5%, ahead of the best open model (Kimi K3) at 94.5% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.