llmrun
Reasoning

ARC-AGI Leaderboard

ARC-AGI tests fluid, abstract reasoning on small visual grid puzzles where each task follows a novel rule the model must infer from a few examples. It deliberately resists memorization and is one of the most-watched measures of general reasoning progress.

Source: epoch19 open models ranked+213 proprietaryData through Sep 2026

Open modelsAll models

All models ranked on ARC-AGI

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScoreVRAM @ Q4
1Claude Fable 5 (xhigh) · proprietary
98.5%
API only
2Claude Fable 5 Max · proprietary
98.5%
API only
3Claude Opus 5.5 (high) · proprietary
98.5%
API only
4Gemini 3.8 Flash (high) · proprietary
98.5%
API only
5GPT 6 Astra (high) · proprietary
98.5%
API only
6GPT 6 Astra (xhigh) · proprietary
98.5%
API only
7GPT 6.1 Sol (high) · proprietary
98.5%
API only
8GPT 6.1 Sol (xhigh) · proprietary
98.5%
API only
9Gemini 3.1 Pro Preview · proprietary
98.0%
API only
10Claude Fable 5.1 Max · proprietary
97.5%
API only
11Claude Opus 5 (high) · proprietary
97.5%
API only
12Claude Opus 5 Max · proprietary
97.5%
API only
13Claude Opus 5.5 (medium) · proprietary
97.5%
API only
14Claude Opus 5.5 (xhigh) · proprietary
97.5%
API only
15Claude Opus 5.5 Max · proprietary
97.5%
API only
16Gemini 3.8 Flash (medium) · proprietary
97.5%
API only
17GPT 5.6 Sol (xhigh) · proprietary
97.5%
API only
18GPT 6 Astra (medium) · proprietary
97.5%
API only
19GPT 6 Astra Max · proprietary
97.5%
API only
20GPT 5.6 Sol (high) · proprietary
97.0%
API only
21Claude Fable 5.1 (xhigh) · proprietary
96.5%
API only
22GPT 5.5 Pro (high) · proprietary
96.5%
API only
23GPT 5.6 Sol Max · proprietary
96.5%
API only
24GPT 5.6 Terra Max · proprietary
96.5%
API only
25GPT 6 Astra (low) · proprietary
96.5%
API only
26GPT 6.1 Sol Max · proprietary
96.5%
API only
27Claude Fable 5.1 (high) · proprietary
96.0%
API only
28Gemini 3 Deep Think Preview · proprietary
96.0%
API only
29Claude Fable 5 (high) · proprietary
95.5%
API only
30Gemini 3.7 Flash (high) · proprietary
95.5%
API only
31GPT 6 Sol Max · proprietary
95.5%
API only
32GPT 6.1 Sol (medium) · proprietary
95.5%
API only
33GPT 5.5 (xhigh) · proprietary
95.0%
API only
34GPT 5.5 Pro (xhigh) · proprietary
95.0%
API only
35Claude Fable 5.1 (medium) · proprietary
94.5%
API only
36DeepSeek V4.1 Flash · 763.2B
94.5%
~458 GB
37GPT 5.4 Pro (Mar 05, 2026, xhigh) · proprietary
94.5%
API only
38GPT 5.5 (high) · proprietary
94.5%
API only
39Kimi K3 · 2779.9B
94.5%
~1674 GB
40Claude Opus 4.6 (120K) · proprietary
94.0%
API only
41GPT 5.6 Terra (xhigh) · proprietary
94.0%
API only
42GPT 5.4 (Mar 05, 2026, xhigh) · proprietary
93.7%
API only
43Claude Opus 4.7 (high) · proprietary
93.5%
API only
44GPT 6.1 Sol (low) · proprietary
93.5%
API only
45GPT 5.4 (Mar 05, 2026, high) · proprietary
92.7%
API only
46GPT 6 Sol (xhigh) · proprietary
92.7%
API only
47Claude Fable 5 (medium) · proprietary
92.5%
API only
48Claude Opus 4.8 Max · proprietary
92.5%
API only
49Gemini 3.5 Flash (high) · proprietary
92.5%
API only
50GPT 5.6 Sol (medium) · proprietary
92.5%
API only
51GPT 5.5 (medium) · proprietary
92.2%
API only
52Claude Opus 4.7 Max · proprietary
92.0%
API only
53Claude Opus 4.8 (high) · proprietary
92.0%
API only
54GPT 5.6 Terra (high) · proprietary
92.0%
API only
55Claude Opus 4.8 (medium) · proprietary
91.5%
API only
56Gemini 3.6 Flash (high) · proprietary
91.2%
API only
57Gemini 3.7 Flash (medium) · proprietary
91.2%
API only
58Claude Opus 4.7 (low) · proprietary
91.0%
API only
59Claude Opus 4.7 (medium) · proprietary
91.0%
API only
60GLM 5.3 Flash · 321.3B
91.0%
~195 GB
61GPT 6 Sol (high) · proprietary
91.0%
API only
62Claude Fable 5 (low) · proprietary
90.5%
API only
63DeepSeek V4 Pro 0813 · 1650.5B
90.5%
~991 GB
64Gemini 3.8 Flash (low) · proprietary
90.5%
API only
65GPT 5.2 Pro (Dec 11, 2025, xhigh) · proprietary
90.5%
API only
66Grok 4.7 (high) · proprietary
90.2%
API only
67Claude Fable 5.1 (low) · proprietary
90.0%
API only
68Grok 4.7 (medium) · proprietary
90.0%
API only
69Grok 4.20 · proprietary
89.5%
API only
70Grok 4.7 (xhigh) · proprietary
89.5%
API only
71DeepSeek V4 Flash 0731 · 304.2B
89.0%
~183 GB
72Claude Opus 5.5 (low) · proprietary
88.5%
API only
73Claude Opus 4.8 (low) · proprietary
88.0%
API only
74GPT 5.6 Luna Max · proprietary
88.0%
API only
75GPT 5.6 Luna (xhigh) · proprietary
87.7%
API only
76Grok 4.6 (medium) · proprietary
87.5%
API only
77Qwen3.8 27B · 27.8B
87.5%
~17 GB
78Grok 4.5 (medium) · proprietary
87.2%
API only
79Grok 4.6 (high) · proprietary
87.0%
API only
80Grok 4.6 (xhigh) · proprietary
87.0%
API only
81GPT 6 Luna Max · proprietary
86.7%
API only
82Claude Sonnet 4.6 (high) · proprietary
86.5%
API only
83GPT 5.2 (Dec 11, 2025, xhigh) · proprietary
86.2%
API only
84GPT 5.4 (Mar 05, 2026, medium) · proprietary
86.2%
API only
85Claude Sonnet 4.6 Max · proprietary
86.0%
API only
86GPT 6 Astra None · proprietary
86.0%
API only
87GPT 5.2 Pro (Dec 11, 2025, high) · proprietary
85.7%
API only
88Grok 4.5 (high) · proprietary
85.7%
API only
89Gemini 3.7 Flash (low) · proprietary
85.2%
API only
90Gemini 3 Flash Preview · proprietary
84.7%
API only
91Inkling Small · 266.0B
84.0%
~160 GB
92GPT 6 Sol (medium) · proprietary
83.7%
API only
93Gemini 3.6 Flash (medium) · proprietary
83.2%
API only
94GPT 5.2 Pro (Dec 11, 2025, medium) · proprietary
81.2%
API only
95Claude Opus 4.5 (Nov 01, 2025, 64K) · proprietary
80.0%
API only
96Inkling · 952.4B
79.5%
~572 GB
97Grok 4.5 (low) · proprietary
79.2%
API only
98GPT 5.2 (Dec 11, 2025, high) · proprietary
78.7%
API only
99GLM 5.2 · 753.3B
77.0%
~456 GB
100GPT 5.6 Terra (medium) · proprietary
77.0%
API only
101Gemini 3.6 Flash (low) · proprietary
76.5%
API only
102GPT 5.6 Luna (high) · proprietary
76.5%
API only
103GPT 5.5 (low) · proprietary
76.2%
API only
104Claude Opus 4.5 (Nov 01, 2025, 32K) · proprietary
75.8%
API only
105Gemini 3 Pro Preview · proprietary
75.0%
API only
106Grok 4.6 (low) · proprietary
74.8%
API only
107GPT 5.6 Sol (low) · proprietary
74.5%
API only
108GPT 6 Luna (xhigh) · proprietary
73.0%
API only
109GPT 5.1 (Nov 13, 2025, high) · proprietary
72.8%
API only
110GPT 5.2 (Dec 11, 2025, medium) · proprietary
72.7%
API only
111GPT 6 Sol (low) · proprietary
72.2%
API only
112Claude Opus 4.5 (Nov 01, 2025, 16K) · proprietary
72.0%
API only
113GPT 6 Luna (high) · proprietary
70.3%
API only
114GPT 5 Pro (Oct 06, 2025, high) · proprietary
70.2%
API only
115GPT 5 Pro (Oct 06, 2025, unspecified) · proprietary
70.2%
API only
116GPT 5.4 (Mar 05, 2026, low) · proprietary
68.2%
API only
117Grok 4 (Jul 09) · proprietary
66.7%
API only
118GPT 5 (Aug 07, 2025, high) · proprietary
65.7%
API only
119Kimi K2.5 · 1026.9B
65.3%
~620 GB
120Claude Sonnet 4.5 (Sep 29, 2025, 32K) · proprietary
63.7%
API only
121GPT 5.4 Mini (Mar 17, 2026, xhigh) · proprietary
63.7%
API only
122MiniMax M2.5 · 228.7B
63.7%
~138 GB
123Grok 4.7 (low) · proprietary
63.3%
API only
124GPT 6 Luna (medium) · proprietary
61.0%
API only
125O3 (Apr 16, 2025, high) · proprietary
60.8%
API only
126GPT 5.6 Terra (low) · proprietary
60.2%
API only
127O3 Pro (Jun 10, 2025, high) · proprietary
59.3%
API only
128Claude Opus 4.5 (Nov 01, 2025, 8K) · proprietary
58.7%
API only
129O4 Mini (Apr 16, 2025, high) · proprietary
58.7%
API only
130GPT 5.4 Mini (Mar 17, 2026, high) · proprietary
58.0%
API only
131GPT 5.1 (Nov 13, 2025, medium) · proprietary
57.7%
API only
132DeepSeek V3.2 · 685.4B
57.0%
~415 GB
133O3 Pro (Jun 10, 2025, medium) · proprietary
57.0%
API only
134GPT 5.6 Luna (medium) · proprietary
56.5%
API only
135GPT 5 (Aug 07, 2025, medium) · proprietary
56.2%
API only
136GPT 5.2 (Dec 11, 2025, low) · proprietary
55.7%
API only
137GPT 5 Mini (Aug 07, 2025, high) · proprietary
54.3%
API only
138O3 (Apr 16, 2025, medium) · proprietary
53.8%
API only
139Gemini 3.5 Flash Lite (high) · proprietary
53.5%
API only
140GPT 5.4 Nano (Mar 17, 2026, xhigh) · proprietary
51.5%
API only
141Gemini 3.5 Flash (minimal) · proprietary
48.8%
API only
142Grok 4 Fast · proprietary
48.5%
API only
143Claude Sonnet 4.5 (Sep 29, 2025, 16K) · proprietary
48.3%
API only
144Claude Haiku 4.5 (Oct 01, 2025, 32K) · proprietary
47.7%
API only
145Claude Sonnet 4.5 (Sep 29, 2025, 8K) · proprietary
46.5%
API only
146GLM 5 · 753.9B
44.7%
~457 GB
147O3 Pro (Jun 10, 2025, low) · proprietary
44.3%
API only
148GPT 5 (Aug 07, 2025, low) · proprietary
44.0%
API only
149O4 Mini (Apr 16, 2025, medium) · proprietary
41.8%
API only
150O3 (Apr 16, 2025, low) · proprietary
41.5%
API only
151Gemini 2.5 Pro (16K) · proprietary
41.0%
API only
152GPT 5.4 Mini (Mar 17, 2026, medium) · proprietary
40.8%
API only
153Claude Opus 4.5 (Nov 01, 2025) · proprietary
40.0%
API only
154Claude Sonnet 4 (May 14, 2025, 16K) · proprietary
40.0%
API only
155Tiny Recursion Model · proprietary
40.0%
API only
156GPT 5.4 Nano (Mar 17, 2026, high) · proprietary
38.2%
API only
157GPT 6 Luna (low) · proprietary
37.7%
API only
158Claude Haiku 4.5 (Oct 01, 2025, 16K) · proprietary
37.3%
API only
159GPT 5 Mini (Aug 07, 2025, medium) · proprietary
37.3%
API only
160Gemini 2.5 Pro (32K) · proprietary
37.0%
API only
161Claude Opus 4 (May 14, 2025, 16K) · proprietary
35.7%
API only
162Gemini 3.6 Flash (minimal) · proprietary
34.5%
API only
163O3 Mini (Jan 31, 2025, high) · proprietary
34.5%
API only
164GPT 5.6 Luna (low) · proprietary
34.2%
API only
165Gemini 2.5 Flash Preview (May 20, 16K) · proprietary
33.3%
API only
166Gemini 2.5 Flash Preview (May 20) · proprietary
33.3%
API only
167GPT 5.1 (Nov 13, 2025, low) · proprietary
33.2%
API only
168Gemini 2.5 Pro Preview (Mar 25) · proprietary
33.0%
API only
169GPT 5.4 Nano (Mar 17, 2026, medium) · proprietary
33.0%
API only
170Gemini 2.5 Flash Preview (May 20, 23K) · proprietary
32.3%
API only
171Gemini 3.5 Flash Lite (medium) · proprietary
32.3%
API only
172Gemini 2.5 Flash Preview (24K thinking) (Apr 17) · proprietary
32.3%
API only
173Gemini 2.5 Pro Preview (Jun 05, 1K) · proprietary
31.3%
API only
174Claude Sonnet 4.5 (Sep 29, 2025, 1K) · proprietary
31.0%
API only
175O1 (Dec 17, 2024, medium) · proprietary
30.7%
API only
176Claude Opus 4 (May 14, 2025, 8K) · proprietary
30.7%
API only
177Gemini 2.5 Pro (8K) · proprietary
29.5%
API only
178GPT 6 Sol None · proprietary
29.3%
API only
179Claude Sonnet 4 (May 14, 2025, 8K) · proprietary
29.0%
API only
180Claude 3.7 Sonnet (Feb 19, 2025, 16K) · proprietary
28.6%
API only
181Claude Sonnet 4 (May 14, 2025, 1K) · proprietary
28.0%
API only
182Codex Mini (May 16, 2025) · proprietary
27.3%
API only
183O1 (Dec 17, 2024, low) · proprietary
27.2%
API only
184Claude Opus 4 (May 14, 2025, 1K) · proprietary
27.0%
API only
185GPT 5 Mini (Aug 07, 2025, low) · proprietary
26.3%
API only
186Gemini 2.5 Flash Preview (May 20, 8K) · proprietary
25.8%
API only
187Claude Haiku 4.5 (Oct 01, 2025, 8K) · proprietary
25.5%
API only
188Claude Sonnet 4.5 (Sep 29, 2025) · proprietary
25.5%
API only
189Claude Sonnet 4 (May 14, 2025) · proprietary
23.8%
API only
190O1 Pro (Mar 19, 2025, low) · proprietary
23.3%
API only
191Claude Opus 4 (May 14, 2025) · proprietary
22.5%
API only
192O3 Mini (Jan 31, 2025, medium) · proprietary
22.3%
API only
193O4 Mini (Apr 16, 2025, low) · proprietary
21.3%
API only
194DeepSeek R1 0528 · 684.5B
21.2%
~415 GB
195Claude 3.7 Sonnet (Feb 19, 2025, 8K) · proprietary
21.2%
API only
196GPT 5 Nano (Aug 07, 2025, medium) · proprietary
20.7%
API only
197GPT 5.4 Nano (Mar 17, 2026, low) · proprietary
18.3%
API only
198O1 Preview (Sep 12, 2024) · proprietary
18.0%
API only
199Gemini 3.5 Flash Lite (low) · proprietary
17.0%
API only
200Claude Haiku 4.5 (Oct 01, 2025, 1K) · proprietary
16.8%
API only
201GPT 5 Nano (Aug 07, 2025, high) · proprietary
16.7%
API only
202Grok 3 Mini (low) · proprietary
16.5%
API only
203Grok 3 Mini Beta (low) · proprietary
16.5%
API only
204Gemini 2.5 Flash Preview (May 20, 1K) · proprietary
16.0%
API only
205DeepSeek R1 · 684.5B
15.8%
~415 GB
206O3 Mini (Jan 31, 2025, low) · proprietary
14.5%
API only
207Claude Haiku 4.5 (Oct 01, 2025) · proprietary
14.3%
API only
208O1 Mini (Sep 12, 2024, medium) · proprietary
14.0%
API only
209O1 Mini (Sep 12, 2024, unspecified) · proprietary
14.0%
API only
210Claude 3.7 Sonnet (Feb 19, 2025) · proprietary
13.6%
API only
211GPT 5.4 Mini (Mar 17, 2026, low) · proprietary
13.0%
API only
212GPT 5.2 (Dec 11, 2025, unspecified) · proprietary
12.3%
API only
213GPT 5.2 2025 12.11 None · proprietary
12.3%
API only
214Claude 3.7 Sonnet (Feb 19, 2025, 1K) · proprietary
11.6%
API only
215Qwen3 235B A22B Instruct 2507 · 235.1B
11.0%
~142 GB
216GPT 4.5 Preview (Feb 27, 2025) · proprietary
10.3%
API only
217GPT 6 Luna None · proprietary
8.8%
API only
218Gemini 3.5 Flash Lite (minimal) · proprietary
7.5%
API only
219Magistral Medium 2506 · proprietary
6.1%
API only
220GPT 5 (Aug 07, 2025, minimal) · proprietary
6.0%
API only
221GPT 5.1 2025 11.13 None · proprietary
5.8%
API only
222GPT 4.1 (Apr 14, 2025) · proprietary
5.5%
API only
223Grok 3 · proprietary
5.5%
API only
224GPT 5 Mini (Aug 07, 2025, minimal) · proprietary
5.3%
API only
225Magistral Small 2506 · 23.6B
5.0%
~15 GB
226GPT 4o (Nov 20, 2024) · proprietary
4.5%
API only
227Llama 4 Maverick 17B 128E Instruct · 401.6B
4.4%
~265 GB
228GPT 5 Nano (Aug 07, 2025, low) · proprietary
4.0%
API only
229GPT 4.1 Mini (Apr 14, 2025) · proprietary
3.5%
API only
230GPT 5 Nano (Aug 07, 2025, minimal) · proprietary
1.5%
API only
231Llama 4 Scout 17B 16E Instruct · 108.6B
0.5%
~72 GB
232GPT 4.1 Nano (Apr 14, 2025) · proprietary
0.0%
API only

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →94.5%0.5%Kimi K3 · 2.8T · 94.5%DeepSeek V4 Pro 0813 · 1.7T · 90.5%Inkling Small · 266B · 84.0%Inkling · 952B · 79.5%GLM 5.2 · 753B · 77.0%Kimi K2.5 · 1T · 65.3%MiniMax M2.5 · 229B · 63.7%DeepSeek V3.2 · 685B · 57.0%GLM 5 · 754B · 44.7%DeepSeek R1 0528 · 685B · 21.2%DeepSeek R1 · 684B · 15.8%Qwen3 235B A22B Instruct 2507 · 235B · 11.0%Llama 4 Maverick 17B 128E Instruct · 402B · 4.4%Llama 4 Scout 17B 16E Instruct · 109B · 0.5%Magistral Small 2506 · 24B · 5.0%Magistral Small 2506Qwen3.8 27B · 28B · 87.5%Qwen3.8 27BDeepSeek V4 Flash 0731 · 304B · 89.0%DeepSeek V4 Flash 0731GLM 5.3 Flash · 321B · 91.0%GLM 5.3 FlashDeepSeek V4.1 Flash · 763B · 94.5%DeepSeek V4.1 Flash
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Magistral Small 2506, 24B, score 5.0% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3.8 27B, 28B, score 87.5% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Flash 0731, 304B, score 89.0% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3 Flash, 321B, score 91.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4.1 Flash, 763B, score 94.5% — on the efficiency frontier (best score at its size or smaller).

ARC-AGI: frequently asked questions

What is the best open LLM on ARC-AGI?
DeepSeek V4.1 Flash is the top open model on ARC-AGI, scoring 94.5%. Among all models tested — including proprietary ones — it ranks #35. The top model overall is Claude Fable 5 (xhigh) (Anthropic) at 98.5%.
What's the best ARC-AGI model you can run on a 24 GB GPU?
Qwen3.8 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (~17 GB), scoring 87.5% on ARC-AGI.
Can open models match proprietary models on ARC-AGI?
Not quite on ARC-AGI: the strongest proprietary model (Claude Fable 5 (xhigh)) scores 98.5%, ahead of the best open model (DeepSeek V4.1 Flash) at 94.5% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.