llmrun
Reasoning

ARC-AGI Leaderboard

ARC-AGI tests fluid, abstract reasoning on small visual grid puzzles where each task follows a novel rule the model must infer from a few examples. It deliberately resists memorization and is one of the most-watched measures of general reasoning progress.

Source: epoch10 open models ranked+148 proprietaryData through Jul 2026

Open modelsAll models

All models ranked on ARC-AGI

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScore
1Gemini 3.1 Pro Preview · proprietary
98.0%
2GPT 5.6 Sol (xhigh) · proprietary
97.5%
3GPT 5.6 Sol (high) · proprietary
97.0%
4GPT 5.5 Pro (high) · proprietary
96.5%
5GPT 5.6 Sol Max · proprietary
96.5%
6GPT 5.6 Terra Max · proprietary
96.5%
7GPT 5.5 (xhigh) · proprietary
95.0%
8GPT 5.5 Pro (xhigh) · proprietary
95.0%
9GPT 5.4 Pro (Mar 05, 2026, xhigh) · proprietary
94.5%
10GPT 5.5 (high) · proprietary
94.5%
11Claude Opus 4.6 (120K) · proprietary
94.0%
12GPT 5.6 Terra (xhigh) · proprietary
94.0%
13GPT 5.4 (Mar 05, 2026, xhigh) · proprietary
93.7%
14Claude Opus 4.7 (high) · proprietary
93.5%
15GPT 5.4 (Mar 05, 2026, high) · proprietary
92.7%
16Claude Opus 4.8 Max · proprietary
92.5%
17Gemini 3.5 Flash (high) · proprietary
92.5%
18GPT 5.6 Sol (medium) · proprietary
92.5%
19GPT 5.5 (medium) · proprietary
92.2%
20Claude Opus 4.7 Max · proprietary
92.0%
21Claude Opus 4.8 (high) · proprietary
92.0%
22GPT 5.6 Terra (high) · proprietary
92.0%
23Claude Opus 4.8 (medium) · proprietary
91.5%
24Claude Opus 4.7 (low) · proprietary
91.0%
25Claude Opus 4.7 (medium) · proprietary
91.0%
26GPT 5.2 Pro (Dec 11, 2025, xhigh) · proprietary
90.5%
27Grok 4.20 · proprietary
89.5%
28Claude Opus 4.8 (low) · proprietary
88.0%
29GPT 5.6 Luna Max · proprietary
88.0%
30GPT 5.6 Luna (xhigh) · proprietary
87.7%
31Gemini 3 Deep Think Preview · proprietary
87.5%
32Claude Sonnet 4.6 (high) · proprietary
86.5%
33GPT 5.2 (Dec 11, 2025, xhigh) · proprietary
86.2%
34GPT 5.4 (Mar 05, 2026, medium) · proprietary
86.2%
35Claude Sonnet 4.6 Max · proprietary
86.0%
36GPT 5.2 Pro (Dec 11, 2025, high) · proprietary
85.7%
37GPT 5.2 Pro (Dec 11, 2025, medium) · proprietary
81.2%
38Claude Opus 4.5 (Nov 01, 2025, 64K) · proprietary
80.0%
39GPT 5.2 (Dec 11, 2025, high) · proprietary
78.7%
40GLM 5.2 (unspecified) · proprietary
77.0%
41GPT 5.6 Terra (medium) · proprietary
77.0%
42GPT 5.6 Luna (high) · proprietary
76.5%
43GPT 5.5 (low) · proprietary
76.2%
44Claude Opus 4.5 (Nov 01, 2025, 32K) · proprietary
75.8%
45Gemini 3 Pro Preview · proprietary
75.0%
46GPT 5.6 Sol (low) · proprietary
74.5%
47GPT 5.1 (Nov 13, 2025, high) · proprietary
72.8%
48GPT 5.2 (Dec 11, 2025, medium) · proprietary
72.7%
49Claude Opus 4.5 (Nov 01, 2025, 16K) · proprietary
72.0%
50GPT 5 Pro (Oct 06, 2025, high) · proprietary
70.2%
51GPT 5 Pro (Oct 06, 2025, unspecified) · proprietary
70.2%
52GPT 5.4 (Mar 05, 2026, low) · proprietary
68.2%
53Grok 4 (Jul 09) · proprietary
66.7%
54GPT 5 (Aug 07, 2025, high) · proprietary
65.7%
55Kimi K2.5 · 1058.6B
65.3%
56Claude Sonnet 4.5 (Sep 29, 2025, 32K) · proprietary
63.7%
57GPT 5.4 Mini (Mar 17, 2026, xhigh) · proprietary
63.7%
58MiniMax M2.5 · 228.7B
63.7%
59O3 (Apr 16, 2025, high) · proprietary
60.8%
60GPT 5.6 Terra (low) · proprietary
60.2%
61O3 Pro (Jun 10, 2025, high) · proprietary
59.3%
62O4 Mini (Apr 16, 2025, high) · proprietary
58.7%
63Claude Opus 4.5 (Nov 01, 2025, 8K) · proprietary
58.7%
64GPT 5.4 Mini (Mar 17, 2026, high) · proprietary
58.0%
65GPT 5.1 (Nov 13, 2025, medium) · proprietary
57.7%
66DeepSeek V3.2 · 685.4B
57.0%
67O3 Pro (Jun 10, 2025, medium) · proprietary
57.0%
68GPT 5.6 Luna (medium) · proprietary
56.5%
69GPT 5 (Aug 07, 2025, medium) · proprietary
56.2%
70GPT 5.2 (Dec 11, 2025, low) · proprietary
55.7%
71GPT 5 Mini (Aug 07, 2025, high) · proprietary
54.3%
72O3 (Apr 16, 2025, medium) · proprietary
53.8%
73GPT 5.4 Nano (Mar 17, 2026, xhigh) · proprietary
51.5%
74Gemini 3.5 Flash (minimal) · proprietary
48.8%
75Grok 4 Fast · proprietary
48.5%
76Claude Sonnet 4.5 (Sep 29, 2025, 16K) · proprietary
48.3%
77Claude Haiku 4.5 (Oct 01, 2025, 32K) · proprietary
47.7%
78Claude Sonnet 4.5 (Sep 29, 2025, 8K) · proprietary
46.5%
79GLM 5 · 753.9B
44.7%
80O3 Pro (Jun 10, 2025, low) · proprietary
44.3%
81GPT 5 (Aug 07, 2025, low) · proprietary
44.0%
82O4 Mini (Apr 16, 2025, medium) · proprietary
41.8%
83O3 (Apr 16, 2025, low) · proprietary
41.5%
84Gemini 2.5 Pro (16K) · proprietary
41.0%
85GPT 5.4 Mini (Mar 17, 2026, medium) · proprietary
40.8%
86Claude Opus 4.5 (Nov 01, 2025) · proprietary
40.0%
87Claude Sonnet 4 (May 14, 2025, 16K) · proprietary
40.0%
88Tiny Recursion Model · proprietary
40.0%
89GPT 5.4 Nano (Mar 17, 2026, high) · proprietary
38.2%
90Claude Haiku 4.5 (Oct 01, 2025, 16K) · proprietary
37.3%
91GPT 5 Mini (Aug 07, 2025, medium) · proprietary
37.3%
92Gemini 2.5 Pro (32K) · proprietary
37.0%
93Claude Opus 4 (May 14, 2025, 16K) · proprietary
35.7%
94O3 Mini (Jan 31, 2025, high) · proprietary
34.5%
95GPT 5.6 Luna (low) · proprietary
34.2%
96Gemini 2.5 Flash Preview (May 20, 16K) · proprietary
33.3%
97Gemini 2.5 Flash Preview (May 20) · proprietary
33.3%
98GPT 5.1 (Nov 13, 2025, low) · proprietary
33.2%
99Gemini 2.5 Pro Preview (Mar 25) · proprietary
33.0%
100GPT 5.4 Nano (Mar 17, 2026, medium) · proprietary
33.0%
101Gemini 2.5 Flash Preview (May 20, 23K) · proprietary
32.3%
102Gemini 2.5 Flash Preview (24K thinking) (Apr 17) · proprietary
32.3%
103Gemini 2.5 Pro Preview (Jun 05, 1K) · proprietary
31.3%
104Claude Sonnet 4.5 (Sep 29, 2025, 1K) · proprietary
31.0%
105Claude Opus 4 (May 14, 2025, 8K) · proprietary
30.7%
106O1 (Dec 17, 2024, medium) · proprietary
30.7%
107Gemini 2.5 Pro (8K) · proprietary
29.5%
108Claude Sonnet 4 (May 14, 2025, 8K) · proprietary
29.0%
109Claude 3.7 Sonnet (Feb 19, 2025, 16K) · proprietary
28.6%
110Claude Sonnet 4 (May 14, 2025, 1K) · proprietary
28.0%
111Codex Mini (May 16, 2025) · proprietary
27.3%
112O1 (Dec 17, 2024, low) · proprietary
27.2%
113Claude Opus 4 (May 14, 2025, 1K) · proprietary
27.0%
114GPT 5 Mini (Aug 07, 2025, low) · proprietary
26.3%
115Gemini 2.5 Flash Preview (May 20, 8K) · proprietary
25.8%
116Claude Haiku 4.5 (Oct 01, 2025, 8K) · proprietary
25.5%
117Claude Sonnet 4.5 (Sep 29, 2025) · proprietary
25.5%
118Claude Sonnet 4 (May 14, 2025) · proprietary
23.8%
119O1 Pro (Mar 19, 2025, low) · proprietary
23.3%
120Claude Opus 4 (May 14, 2025) · proprietary
22.5%
121O3 Mini (Jan 31, 2025, medium) · proprietary
22.3%
122Gemini 3 Flash Preview · proprietary
21.5%
123O4 Mini (Apr 16, 2025, low) · proprietary
21.3%
124Claude 3.7 Sonnet (Feb 19, 2025, 8K) · proprietary
21.2%
125DeepSeek R1 0528 · 684.5B
21.2%
126GPT 5 Nano (Aug 07, 2025, medium) · proprietary
20.7%
127GPT 5.4 Nano (Mar 17, 2026, low) · proprietary
18.3%
128O1 Preview (Sep 12, 2024) · proprietary
18.0%
129Claude Haiku 4.5 (Oct 01, 2025, 1K) · proprietary
16.8%
130GPT 5 Nano (Aug 07, 2025, high) · proprietary
16.7%
131Grok 3 Mini (low) · proprietary
16.5%
132Grok 3 Mini Beta (low) · proprietary
16.5%
133Gemini 2.5 Flash Preview (May 20, 1K) · proprietary
16.0%
134DeepSeek R1 · 684.5B
15.8%
135O3 Mini (Jan 31, 2025, low) · proprietary
14.5%
136Claude Haiku 4.5 (Oct 01, 2025) · proprietary
14.3%
137O1 Mini (Sep 12, 2024, medium) · proprietary
14.0%
138O1 Mini (Sep 12, 2024, unspecified) · proprietary
14.0%
139Claude 3.7 Sonnet (Feb 19, 2025) · proprietary
13.6%
140GPT 5.4 Mini (Mar 17, 2026, low) · proprietary
13.0%
141GPT 5.2 (Dec 11, 2025, unspecified) · proprietary
12.3%
142Claude 3.7 Sonnet (Feb 19, 2025, 1K) · proprietary
11.6%
143Qwen3 235B A22B Instruct 2507 · 235.1B
11.0%
144GPT 4.5 Preview (Feb 27, 2025) · proprietary
10.3%
145GPT 5 (Aug 07, 2025, minimal) · proprietary
6.0%
146Magistral Medium 2506 · proprietary
5.9%
147GPT 5.1 2025 11.13 None · proprietary
5.8%
148GPT 4.1 (Apr 14, 2025) · proprietary
5.5%
149Grok 3 · proprietary
5.5%
150GPT 5 Mini (Aug 07, 2025, minimal) · proprietary
5.3%
151Magistral Small 2506 · 23.6B
5.0%
152GPT 4o (Nov 20, 2024) · proprietary
4.5%
153Llama 4 Maverick 17B 128E Instruct · 401.6B
4.4%
154GPT 5 Nano (Aug 07, 2025, low) · proprietary
4.0%
155GPT 4.1 Mini (Apr 14, 2025) · proprietary
3.5%
156GPT 5 Nano (Aug 07, 2025, minimal) · proprietary
1.5%
157Llama 4 Scout 17B 16E Instruct · 108.6B
0.5%
158GPT 4.1 Nano (Apr 14, 2025) · proprietary
0.0%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →65.3%0.5%DeepSeek V3.2 · 685B · 57.0%GLM 5 · 754B · 44.7%DeepSeek R1 0528 · 685B · 21.2%DeepSeek R1 · 685B · 15.8%Qwen3 235B A22B Instruct 2507 · 235B · 11.0%Llama 4 Maverick 17B 128E Instruct · 402B · 4.4%Llama 4 Scout 17B 16E Instruct · 109B · 0.5%Magistral Small 2506 · 24B · 5.0%Magistral Small 2506MiniMax M2.5 · 229B · 63.7%MiniMax M2.5Kimi K2.5 · 1.1T · 65.3%Kimi K2.5
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Magistral Small 2506, 24B, score 5.0% — on the efficiency frontier (best score at its size or smaller).
  • MiniMax M2.5, 229B, score 63.7% — on the efficiency frontier (best score at its size or smaller).
  • Kimi K2.5, 1.1T, score 65.3% — on the efficiency frontier (best score at its size or smaller).

ARC-AGI: frequently asked questions

What is the best open LLM on ARC-AGI?
Kimi K2.5 is the top open model on ARC-AGI, scoring 65.3%. Among all models tested — including proprietary ones — it ranks #55. The top model overall is Gemini 3.1 Pro Preview (Google DeepMind) at 98.0%.
What's the best ARC-AGI model you can run on a 24 GB GPU?
Magistral Small 2506 is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 13 GB), scoring 5.0% on ARC-AGI.
Can open models match proprietary models on ARC-AGI?
Not quite on ARC-AGI: the strongest proprietary model (Gemini 3.1 Pro Preview) scores 98.0%, ahead of the best open model (Kimi K2.5) at 65.3% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.