llmrun
Math

AIME 2024/2025 Leaderboard

AIME (American Invitational Mathematics Examination) is a prestigious high-school competition of hard, integer-answer problems. It's a widely-cited yardstick for multi-step mathematical reasoning, here on the 2024–2025 papers.

Source: epoch86 open models ranked+197 proprietaryData through Oct 2026

Open modelsAll models

All models ranked on AIME 2024/2025

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScoreVRAM @ Q4
1Claude Fable 5 (high) · proprietary
100.0%
API only
2Claude Fable 5.1 Max · proprietary
100.0%
API only
3Claude Opus 5.5 Max · proprietary
100.0%
API only
4Claude Sonnet 5.5 Max · proprietary
100.0%
API only
5Gemini 4 Argon (high) · proprietary
100.0%
API only
6GPT 5.5 Pre Release (xhigh) · proprietary
100.0%
API only
7GPT 5.5 Pro Pre Release (xhigh) · proprietary
100.0%
API only
8GPT 5.6 Sol Max · proprietary
100.0%
API only
9GPT 6 Astra Max · proprietary
100.0%
API only
10GPT 6 Sol Max · proprietary
100.0%
API only
11GPT 6.1 Sol Max · proprietary
100.0%
API only
12Qwen3.8 Max (Sep 02, xhigh) · proprietary
100.0%
API only
13Claude Fable 5 Max · proprietary
99.7%
API only
14GPT 5.6 Terra Max · proprietary
99.7%
API only
15Muse Wonder 1.0 Max · proprietary
99.4%
API only
16Qwen3.8 Max (xhigh) · proprietary
99.4%
API only
17Grok 4.6 (xhigh) · proprietary
99.2%
API only
18Muse Spark 1.3 (xhigh) · proprietary
99.2%
API only
19Claude Haiku 5.5 (xhigh) · proprietary
98.9%
API only
20Claude Opus 5 Max · proprietary
98.9%
API only
21Gemini 3.8 Flash (high) · proprietary
98.9%
API only
22GPT 6 Luna Max · proprietary
98.9%
API only
23DeepSeek V4 Pro 0813 · 1650.5B
98.6%
~991 GB
24Claude Opus 4.8 Max · proprietary
98.3%
API only
25DeepSeek V4.1 Flash · 763.2B
98.3%
~458 GB
26GPT 5.6 Luna Max · proprietary
98.3%
API only
27Mistral Large 4.0 (high) · proprietary
98.3%
API only
28Grok 4.7 (xhigh) · proprietary
98.1%
API only
29Muse Spark Open 816B A42B · 822.0B
98.1%
~543 GB
30Claude Opus 4.7 (xhigh) · proprietary
97.8%
API only
31Claude Fable 5 (low) · proprietary
97.8%
API only
32Claude Opus 4.8 (low) · proprietary
97.8%
API only
33Claude Opus 5 · proprietary
97.8%
API only
34GPT 5.4 (Mar 05, 2026, high) · proprietary
97.8%
API only
35Grok 4.5 (high) · proprietary
97.8%
API only
36Grok 4.6 (high) · proprietary
97.8%
API only
37Claude Haiku 5.5 (high) · proprietary
97.2%
API only
38Gemini 3.7 Flash (high) · proprietary
97.2%
API only
39Kimi K3 · 2779.9B
97.2%
~1674 GB
40DeepSeek V4 Pro · 1598.8B
96.7%
~960 GB
41GPT 5.2 (Dec 11, 2025, high) · proprietary
96.1%
API only
42Kimi K2.6 · 1026.9B
96.1%
~620 GB
43GPT 5.2 (Dec 11, 2025, xhigh) · proprietary
96.1%
API only
44Gemini 3.1 Pro Preview · proprietary
95.6%
API only
45Gemini 3 Flash Preview (high) · proprietary
95.6%
API only
46Gemini 3.1 Pro Preview (high) · proprietary
95.6%
API only
47Gemini 3.5 Flash (high) · proprietary
95.6%
API only
48GPT 5.4 (Mar 05, 2026, medium) · proprietary
95.6%
API only
49GPT 5.6 Sol (low) · proprietary
95.6%
API only
50Kimi K2.7 Code · 1026.9B
95.6%
~620 GB
51Qwen3.7 Max · proprietary
95.6%
API only
52GPT 5.4 (Mar 05, 2026, xhigh) · proprietary
95.3%
API only
53Claude Sonnet 5 (xhigh) · proprietary
94.7%
API only
54Claude Opus 4.6 (64K) · proprietary
94.4%
API only
55DeepSeek V4 Flash 0731 · 304.2B
94.4%
~183 GB
56Gemini 3.6 Flash (high) · proprietary
94.2%
API only
57GLM 5.3 Flash · 321.3B
93.9%
~195 GB
58GPT 5.2 (Dec 11, 2025, medium) · proprietary
93.9%
API only
59Claude Opus 5 (low) · proprietary
93.3%
API only
60GLM 5.1 · 753.9B
93.3%
~457 GB
61Grok 4.3 (high) · proprietary
93.3%
API only
62Qwen3.6 Plus · proprietary
93.3%
API only
63Qwen3.7 Plus · proprietary
93.3%
API only
64Claude Opus 4.6 (32K) · proprietary
93.1%
API only
65Gemini 3 Flash Preview · proprietary
92.8%
API only
66Grok 4.20 0309 Reasoning · proprietary
92.2%
API only
67Kimi K2.5 · 1026.9B
92.2%
~620 GB
68Gemini 3 Pro Preview · proprietary
91.4%
API only
69GPT 5 (Aug 07, 2025, high) · proprietary
91.4%
API only
70Claude Opus 4.6 Max · proprietary
91.1%
API only
71GLM 5.3 · 753.3B
91.1%
~456 GB
72Qwen3.6 27B · 27.8B
91.1%
~17 GB
73Qwen3.6 Max Preview · proprietary
91.1%
API only
74Inkling Small · 266.0B
90.0%
~160 GB
75Muse Spark · proprietary
88.9%
API only
76Gemini 3.5 Flash (low) · proprietary
88.9%
API only
77GPT 5.4 Mini (Mar 17, 2026, xhigh) · proprietary
88.9%
API only
78GPT 5.6 Terra (low) · proprietary
88.9%
API only
79GPT OSS 120B · 116.8B
88.9%
~71 GB
80Inkling · 952.4B
88.9%
~572 GB
81Qwen3.5 397B A17B · 403.4B
88.9%
~242 GB
82GPT 5.1 (Nov 13, 2025, high) · proprietary
88.6%
API only
83DeepSeek Reasoner · proprietary
87.8%
API only
84GPT 5.4 Nano (Mar 17, 2026, high) · proprietary
87.8%
API only
85GPT 5 (Aug 07, 2025, medium) · proprietary
87.2%
API only
86GPT 5.4 Mini (Mar 17, 2026, high) · proprietary
87.2%
API only
87Claude Opus 4.7 Max · proprietary
86.7%
API only
88GPT 5 Mini (Aug 07, 2025, high) · proprietary
86.7%
API only
89NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B
86.7%
~370 GB
90Qwen3 235B A22B Thinking 2507 · 235.1B
86.7%
~142 GB
91Qwen3.5 Plus · proprietary
86.7%
API only
92Qwen3.6 35B A3B · 36.0B
86.7%
~22 GB
93Qwen3.7 Flash · proprietary
86.7%
API only
94GLM 5.2 · 753.3B
86.4%
~456 GB
95Claude Opus 4.5 (Nov 01, 2025, 32K) · proprietary
86.1%
API only
96Claude Sonnet 4.6 (32K) · proprietary
85.8%
API only
97GPT 5.1 (Nov 13, 2025, medium) · proprietary
85.6%
API only
98Claude Opus 4.8 None · proprietary
84.4%
API only
99GPT 5.4 (Mar 05, 2026, low) · proprietary
84.4%
API only
100GPT 5.5 (low) · proprietary
84.4%
API only
101O3 (Apr 16, 2025, medium) · proprietary
84.4%
API only
102Qwen3.5 Flash · proprietary
84.4%
API only
103Qwen3.6 Flash · proprietary
84.4%
API only
104Gemini 2.5 Pro · proprietary
84.2%
API only
105Grok 4 (Jul 09) · proprietary
84.0%
API only
106O3 (Apr 16, 2025, high) · proprietary
83.9%
API only
107GLM 4.7 · 358.3B
83.3%
~216 GB
108Kimi K2 Thinking · 1026.4B
83.1%
~620 GB
109Claude Sonnet 4.6 (medium) · proprietary
82.2%
API only
110Gemini 3.6 Flash (low) · proprietary
82.2%
API only
111Gemma 4 26B A4B IT · 25.8B
82.2%
~16 GB
112Claude Opus 4.5 (Nov 01, 2025, 16K) · proprietary
81.7%
API only
113O4 Mini (Apr 16, 2025, high) · proprietary
81.7%
API only
114GPT 5 Nano (Aug 07, 2025, high) · proprietary
81.1%
API only
115Claude Sonnet 5 Max · proprietary
80.0%
API only
116Gemini 3.1 Flash Lite (high) · proprietary
80.0%
API only
117Gemini 3.5 Flash (minimal) · proprietary
80.0%
API only
118Gemini 3.6 Flash (minimal) · proprietary
80.0%
API only
119GLM 5 · 753.9B
80.0%
~457 GB
120Qwen3.7 Plus None · proprietary
80.0%
API only
121GPT 5.2 (Dec 11, 2025, low) · proprietary
78.9%
API only
122GPT 5 Mini (Aug 07, 2025, medium) · proprietary
78.3%
API only
123Claude Sonnet 4.5 (Sep 29, 2025, 32K) · proprietary
77.8%
API only
124Claude Sonnet 4.5 (Sep 29, 2025, 59K) · proprietary
77.8%
API only
125Grok 3 Mini Beta (high) · proprietary
77.8%
API only
126Qwen3.7 Flash None · proprietary
77.8%
API only
127O3 Mini (Jan 31, 2025, high) · proprietary
76.9%
API only
128Claude Sonnet 4.6 (high) · proprietary
75.6%
API only
129GPT 5 Nano (Aug 07, 2025, medium) · proprietary
74.2%
API only
130Gemma 4 31B IT · 31.3B
73.3%
~20 GB
131O1 (Dec 17, 2024, high) · proprietary
73.3%
API only
132O1 (Dec 17, 2024, medium) · proprietary
73.3%
API only
133O4 Mini (Apr 16, 2025, medium) · proprietary
73.3%
API only
134Qwen3 Max (Sep 23, 2025) · proprietary
73.3%
API only
135Gemini 2.5 Flash Preview (Apr 17) · proprietary
73.1%
API only
136Claude Sonnet 4 (May 14, 2025, 32K) · proprietary
71.1%
API only
137Claude Sonnet 4.5 (Sep 29, 2025, 16K) · proprietary
71.1%
API only
138Claude Sonnet 4.6 Max · proprietary
71.1%
API only
139Gemini 3.5 Flash Lite (high) · proprietary
71.1%
API only
140MiniMax M3 · 427.0B
71.1%
~257 GB
141Gemini 2.5 Flash Preview (May 20) · proprietary
70.8%
API only
142Qwen3 30B A3B Thinking 2507 · 30.5B
70.3%
~19 GB
143Qwen3.5 35B A3B · 36.0B
70.0%
~22 GB
144Claude Opus 4.1 (Aug 05, 2025, 27K) · proprietary
68.9%
API only
145Claude Sonnet 4 (May 14, 2025, 59K) · proprietary
68.9%
API only
146GPT 5.4 Nano (Mar 17, 2026, low) · proprietary
68.9%
API only
147GPT 5.6 Sol None · proprietary
68.9%
API only
148GPT 5.5 Instant · proprietary
68.1%
API only
149Seed OSS 36B Instruct · 36.2B
67.5%
~22 GB
150Qwen3 32B · 32.8B
66.9%
~20 GB
151Claude Haiku 4.5 (Oct 01, 2025, 32K) · proprietary
66.7%
API only
152GPT 5.6 Luna (low) · proprietary
66.7%
API only
153DeepSeek R1 0528 · 684.5B
66.4%
~415 GB
154Qwen3 14B · 14.8B
66.4%
~9.5 GB
155GPT OSS 20B · 20.9B
65.3%
~13 GB
156Claude Opus 4 (May 14, 2025, 27K) · proprietary
64.4%
API only
157Claude Opus 4.1 (Aug 05, 2025, 16K) · proprietary
64.4%
API only
158GPT 5.1 (Nov 13, 2025, low) · proprietary
63.9%
API only
159O3 Mini (Jan 31, 2025, medium) · proprietary
63.9%
API only
160Qwen3 30B A3B · 30.5B
62.8%
~19 GB
161GPT 5.2 2025 12.11 None · proprietary
62.2%
API only
162Grok 3 Mini Beta (low) · proprietary
62.2%
API only
163Qwen3 30B A3B Instruct 2507 · 30.5B
62.2%
~19 GB
164Qwen3.5 9B · 9.7B
61.7%
~6.4 GB
165Claude Opus 4 (May 14, 2025, 16K) · proprietary
60.0%
API only
166Gemini 3.5 Flash Lite (low) · proprietary
60.0%
API only
167O3 (Apr 16, 2025, low) · proprietary
60.0%
API only
168QwQ 32B · 32.8B
59.2%
~21 GB
169GLM 4.7 Flash · 31.2B
58.3%
~20 GB
170Claude 3.7 Sonnet (Feb 19, 2025, 64K) · proprietary
57.8%
API only
171Gemini 2.0 Flash Thinking Exp (Jan 21) · proprietary
57.8%
API only
172GPT 5.4 2026 03.05 None · proprietary
57.8%
API only
173GPT 5.5 None · proprietary
57.8%
API only
174O4 Mini (Apr 16, 2025, low) · proprietary
57.8%
API only
175Qwen3 8B · 8.2B
56.1%
~5.5 GB
176Qwen3.5 4B · 4.7B
55.8%
~3.3 GB
177DeepSeek R1 Distill Qwen 32B · 32.8B
55.6%
~21 GB
178GPT 5 Mini (Aug 07, 2025, minimal) · proprietary
55.6%
API only
179Grok 3 Beta · proprietary
55.6%
API only
180Claude 3.7 Sonnet (Feb 19, 2025, 32K) · proprietary
53.3%
API only
181Claude Sonnet 4 (May 14, 2025, 16K) · proprietary
53.3%
API only
182DeepSeek R1 · 684.5B
53.3%
~415 GB
183GPT 5.6 Terra None · proprietary
53.3%
API only
184O1 (Dec 17, 2024, low) · proprietary
53.3%
API only
185Qwen3 4B Instruct 2507 · 4.0B
52.2%
~2.9 GB
186DeepSeek R1 Distill Llama 70B · 70.6B
51.4%
~43 GB
187Gemini 3.5 Flash Lite (minimal) · proprietary
51.1%
API only
188DeepSeek R1 Distill Qwen 14B · 14.8B
50.6%
~9.6 GB
189DeepSeek Chat · proprietary
48.9%
API only
190Claude Opus 4.5 (Nov 01, 2025) · proprietary
48.1%
API only
191O1 Mini (Sep 12, 2024, high) · proprietary
46.9%
API only
192Claude 3.7 Sonnet (Feb 19, 2025, 16K) · proprietary
46.7%
API only
193GPT 5 (Aug 07, 2025, minimal) · proprietary
46.7%
API only
194GPT 5 Nano (Aug 07, 2025, low) · proprietary
46.7%
API only
195GPT 5.4 Nano 2026 03.17 None · proprietary
46.7%
API only
196GPT 4.1 Mini (Apr 14, 2025) · proprietary
44.7%
API only
197O1 Mini (Sep 12, 2024, medium) · proprietary
44.7%
API only
198Gemini 3.1 Flash Lite (low) · proprietary
44.4%
API only
199O3 Mini (Jan 31, 2025, low) · proprietary
44.4%
API only
200DeepSeek R1 0528 Qwen3 8B · 8.2B
43.9%
~5.5 GB
201Claude Opus 4 (May 14, 2025) · proprietary
42.2%
API only
202Claude Opus 4.1 (Aug 05, 2025) · proprietary
40.0%
API only
203GPT 5.6 Luna None · proprietary
40.0%
API only
204GPT 4.1 (Apr 14, 2025) · proprietary
38.3%
API only
205DeepSeek v3 0324 · 684.5B
37.8%
~415 GB
206Gemini 3.1 Flash Lite (minimal) · proprietary
37.8%
API only
207GPT 4.5 Preview (Feb 27, 2025) · proprietary
37.8%
API only
208GPT 5.1 2025 11.13 None · proprietary
37.8%
API only
209Claude Haiku 4.5 (Oct 01, 2025) · proprietary
35.8%
API only
210Claude Sonnet 4.5 (Sep 29, 2025) · proprietary
35.6%
API only
211GPT 5 Nano (Aug 07, 2025, minimal) · proprietary
35.6%
API only
212Mistral Medium 2505 · proprietary
32.2%
API only
213Gemini 2.0 Flash 001 · proprietary
31.1%
API only
214O1 Preview (Sep 12, 2024) · proprietary
31.1%
API only
215Mistral Small 3.2 24B Instruct 2506 · 24.0B
30.3%
~15 GB
216Magistral Small 2506 · 23.6B
30.0%
~15 GB
217Claude Sonnet 4 (May 14, 2025) · proprietary
28.9%
API only
218GPT 4.1 Nano (Apr 14, 2025) · proprietary
28.9%
API only
219Magistral Small 2509 · 24.0B
28.1%
~15 GB
220GPT 5.4 Mini 2026 03.17 None · proprietary
26.7%
API only
221Gemini 1.5 Pro 002 · proprietary
23.1%
API only
222Gemma 3 27B IT · 27.4B
22.5%
~18 GB
223Claude 3.7 Sonnet (Feb 19, 2025) · proprietary
21.9%
API only
224DeepSeek R1 Distill Qwen 1.5B · 1.8B
21.4%
~1.4 GB
225Llama 4 Maverick 17B 128E Instruct · 401.6B
20.6%
~265 GB
226Qwen Plus (Jan 25, 2025) · proprietary
17.8%
API only
227Gemma 3 12B IT · 12.2B
16.7%
~8.0 GB
228Gemini 1.5 Flash 002 · proprietary
16.3%
API only
229Qwen Max (Jan 25, 2025) · proprietary
16.1%
API only
230DeepSeek v3 · 684.5B
15.8%
~415 GB
231Phi 4 · 14.7B
13.8%
~9.5 GB
232Grok 2 (Dec 12) · proprietary
11.5%
API only
233Llama 3.1 405B Instruct · 405.9B
9.7%
~268 GB
234Claude 3.5 Sonnet (Oct 22, 2024) · proprietary
8.5%
API only
235Mistral Large Instruct 2407 · 122.6B
8.5%
~81 GB
236Qwen2.5 72B Instruct · 72.7B
8.1%
~45 GB
237Qwen3 1.7B · 2.0B
8.1%
~1.8 GB
238Llama 4 Scout 17B 16E Instruct · 108.6B
7.8%
~72 GB
239Mistral Large Instruct 2411 · 122.6B
7.8%
~75 GB
240Gemma 3 4B IT · 4.3B
7.5%
~2.8 GB
241Qwen2.5 32B Instruct · 32.8B
7.4%
~21 GB
242GPT 4o Mini (Jul 18, 2024) · proprietary
6.9%
API only
243Gemini 1.5 Pro 001 · proprietary
6.8%
API only
244GPT 4 Turbo (Apr 09, 2024) · proprietary
6.7%
API only
245Mistral Small 24B Instruct 2501 · 23.6B
6.7%
~15 GB
246Claude 3.5 Sonnet (Jun 20, 2024) · proprietary
6.5%
API only
247GPT 4o (Aug 06, 2024) · proprietary
6.4%
API only
248GPT 4o (May 13, 2024) · proprietary
6.3%
API only
249GPT 4o (Nov 20, 2024) · proprietary
6.3%
API only
250Qwen Turbo (Nov 01, 2024) · proprietary
6.1%
API only
251Mistral Small 3.1 24B Instruct 2503 · 24.0B
5.8%
~15 GB
252Llama 3.3 70B Instruct · 70.6B
5.1%
~47 GB
253Claude 3 Opus (Feb 29, 2024) · proprietary
4.7%
API only
254Gemini 1.5 Flash 8B 001 · proprietary
4.6%
API only
255Llama 3.1 Tulu 3 70B DPO · 70.6B
4.4%
~43 GB
256Claude 3.5 Haiku (Oct 22, 2024) · proprietary
4.3%
API only
257Meta Llama 3 70B Instruct · 70.6B
4.3%
~47 GB
258Gemini 1.5 Flash 001 · proprietary
3.9%
API only
259Llama 3.1 70B Instruct · 70.6B
3.6%
~47 GB
260Granite 4.0 Micro · 3.4B
2.8%
~2.5 GB
261Llama 3.2 90B Vision Instruct · 88.6B
2.6%
~195 GB
262Claude 2.0 · proprietary
2.5%
API only
263Claude 3 Sonnet (Feb 29, 2024) · proprietary
2.5%
API only
264Hermes 2 Theta Llama 3 70B · 70.6B
2.5%
~43 GB
265Qwen2.5 7B Instruct · 7.6B
2.5%
~5.0 GB
266GPT 3.5 Turbo (Jan 25) · proprietary
2.2%
API only
267Claude 2.1 · proprietary
1.9%
API only
268Meta Llama 3 8B Instruct · 8.0B
1.9%
~5.3 GB
269Mistral Large 2402 · proprietary
1.9%
API only
270Claude 3 Haiku (Mar 07, 2024) · proprietary
1.8%
API only
271Llama 3.1 8B Instruct · 8.0B
1.7%
~5.3 GB
272Gemma 2 27B IT · 27.2B
1.4%
~18 GB
273Gemini 1.0 Pro 001 · proprietary
1.1%
API only
274Gemma 3 1B IT · 1000M
1.1%
~0.7 GB
275GPT 4 (Jun 13) · proprietary
1.1%
API only
276Deepseek Llm 67B Chat · 67B
0.8%
~41 GB
277Granite 4.0 1B · 1.6B
0.8%
~1.4 GB
278Gemma 2 9B IT · 9.2B
0.6%
~6.1 GB
279GPT 4 (Mar 14) · proprietary
0.6%
API only
280Llama 3.2 1B Instruct · 1.2B
0.6%
~0.8 GB
281Granite 4.0 350M · 352M
0.3%
~0.6 GB
282Mistral 7B Instruct v0.3 · 7.2B
0.3%
~4.9 GB
283Llama 2 70B Chat HF · 69.0B
0.0%
~152 GB

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

1B10B100B1Tmodel size (log scale) →98.6%0.0%Muse Spark Open 816B A42B · 822B · 98.1%Kimi K3 · 2.8T · 97.2%DeepSeek V4 Pro · 1.6T · 96.7%Kimi K2.6 · 1T · 96.1%Kimi K2.7 Code · 1T · 95.6%GLM 5.3 Flash · 321B · 93.9%GLM 5.1 · 754B · 93.3%Kimi K2.5 · 1T · 92.2%GLM 5.3 · 753B · 91.1%Inkling Small · 266B · 90.0%GPT OSS 120B · 117B · 88.9%Qwen3.5 397B A17B · 403B · 88.9%Inkling · 952B · 88.9%NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 561B · 86.7%Qwen3 235B A22B Thinking 2507 · 235B · 86.7%Qwen3.6 35B A3B · 36B · 86.7%GLM 5.2 · 753B · 86.4%GLM 4.7 · 358B · 83.3%Kimi K2 Thinking · 1T · 83.1%GLM 5 · 754B · 80.0%Gemma 4 31B IT · 31B · 73.3%MiniMax M3 · 427B · 71.1%Qwen3 30B A3B Thinking 2507 · 31B · 70.3%Qwen3.5 35B A3B · 36B · 70.0%Seed OSS 36B Instruct · 36B · 67.5%Qwen3 32B · 33B · 66.9%DeepSeek R1 0528 · 685B · 66.4%GPT OSS 20B · 21B · 65.3%Qwen3 30B A3B · 31B · 62.8%Qwen3 30B A3B Instruct 2507 · 31B · 62.2%QwQ 32B · 33B · 59.2%GLM 4.7 Flash · 31B · 58.3%DeepSeek R1 Distill Qwen 32B · 33B · 55.6%DeepSeek R1 · 684B · 53.3%DeepSeek R1 Distill Llama 70B · 71B · 51.4%DeepSeek R1 Distill Qwen 14B · 15B · 50.6%DeepSeek R1 0528 Qwen3 8B · 8B · 43.9%DeepSeek v3 0324 · 685B · 37.8%Mistral Small 3.2 24B Instruct 2506 · 24B · 30.3%Magistral Small 2506 · 24B · 30.0%Magistral Small 2509 · 24B · 28.1%Gemma 3 27B IT · 27B · 22.5%Llama 4 Maverick 17B 128E Instruct · 402B · 20.6%Gemma 3 12B IT · 12B · 16.7%DeepSeek v3 · 685B · 15.8%Phi 4 · 15B · 13.8%Llama 3.1 405B Instruct · 406B · 9.7%Mistral Large Instruct 2407 · 123B · 8.5%Qwen2.5 72B Instruct · 73B · 8.1%Qwen3 1.7B · 2B · 8.1%Llama 4 Scout 17B 16E Instruct · 109B · 7.8%Mistral Large Instruct 2411 · 123B · 7.8%Gemma 3 4B IT · 4B · 7.5%Qwen2.5 32B Instruct · 33B · 7.4%Mistral Small 24B Instruct 2501 · 24B · 6.7%Mistral Small 3.1 24B Instruct 2503 · 24B · 5.8%Llama 3.3 70B Instruct · 71B · 5.1%Llama 3.1 Tulu 3 70B DPO · 71B · 4.4%Meta Llama 3 70B Instruct · 71B · 4.3%Llama 3.1 70B Instruct · 71B · 3.6%Granite 4.0 Micro · 3B · 2.8%Llama 3.2 90B Vision Instruct · 89B · 2.6%Hermes 2 Theta Llama 3 70B · 71B · 2.5%Qwen2.5 7B Instruct · 8B · 2.5%Meta Llama 3 8B Instruct · 8B · 1.9%Llama 3.1 8B Instruct · 8B · 1.7%Gemma 2 27B IT · 27B · 1.4%Deepseek Llm 67B Chat · 67B · 0.8%Granite 4.0 1B · 2B · 0.8%Gemma 2 9B IT · 9B · 0.6%Llama 3.2 1B Instruct · 1B · 0.6%Mistral 7B Instruct v0.3 · 7B · 0.3%Llama 2 70B Chat HF · 69B · 0.0%Granite 4.0 350M · 352M · 0.3%Granite 4.0 350MGemma 3 1B IT · 1000M · 1.1%Gemma 3 1B ITDeepSeek R1 Distill Qwen 1.5B · 2B · 21.4%DeepSeek R1 Distill Q…Qwen3 4B Instruct 2507 · 4B · 52.2%Qwen3.5 4B · 5B · 55.8%Qwen3 8B · 8B · 56.1%Qwen3 8BQwen3.5 9B · 10B · 61.7%Qwen3.5 9BQwen3 14B · 15B · 66.4%Qwen3 14BGemma 4 26B A4B IT · 26B · 82.2%Gemma 4 26B A4B ITQwen3.6 27B · 28B · 91.1%Qwen3.6 27BDeepSeek V4 Flash 0731 · 304B · 94.4%DeepSeek V4.1 Flash · 763B · 98.3%DeepSeek V4.1 FlashDeepSeek V4 Pro 0813 · 1.7T · 98.6%DeepSeek V4 Pro 0813
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Granite 4.0 350M, 352M, score 0.3% — on the efficiency frontier (best score at its size or smaller).
  • Gemma 3 1B IT, 1000M, score 1.1% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek R1 Distill Qwen 1.5B, 2B, score 21.4% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3 4B Instruct 2507, 4B, score 52.2% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3.5 4B, 5B, score 55.8% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3 8B, 8B, score 56.1% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3.5 9B, 10B, score 61.7% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3 14B, 15B, score 66.4% — on the efficiency frontier (best score at its size or smaller).
  • Gemma 4 26B A4B IT, 26B, score 82.2% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3.6 27B, 28B, score 91.1% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Flash 0731, 304B, score 94.4% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4.1 Flash, 763B, score 98.3% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Pro 0813, 1.7T, score 98.6% — on the efficiency frontier (best score at its size or smaller).

AIME 2024/2025: frequently asked questions

What is the best open LLM on AIME 2024/2025?
DeepSeek V4 Pro 0813 is the top open model on AIME 2024/2025, scoring 98.6%. Among all models tested — including proprietary ones — it ranks #23. The top model overall is Claude Opus 5.5 Max (Anthropic) at 100.0%.
What's the best AIME 2024/2025 model you can run on a 24 GB GPU?
Qwen3.6 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (~17 GB), scoring 91.1% on AIME 2024/2025.
What's the best AIME 2024/2025 model you can run on a 12 GB GPU?
Qwen3 14B is the highest-scoring open model that fits in 12 GB at 4-bit quantization (~9.5 GB), scoring 66.4% on AIME 2024/2025.
Can open models match proprietary models on AIME 2024/2025?
Not quite on AIME 2024/2025: the strongest proprietary model (Claude Opus 5.5 Max) scores 100.0%, ahead of the best open model (DeepSeek V4 Pro 0813) at 98.6% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.