Math
MATH Level 5 Leaderboard
MATH Level 5 covers the hardest tier of competition-mathematics problems, testing multi-step symbolic and quantitative reasoning.
Source: epoch39 open models ranked+68 proprietaryData through Oct 2025
All models ranked on MATH Level 5
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 | GPT 5 (Aug 07, 2025, high) · proprietary | 98.1% | API only |
| 2 | GPT 5 (Aug 07, 2025, medium) · proprietary | 97.9% | API only |
| 3 | GPT 5 Mini (Aug 07, 2025, high) · proprietary | 97.9% | API only |
| 4 | O4 Mini (Apr 16, 2025, high) · proprietary | 97.8% | API only |
| 5 | O3 (Apr 16, 2025, high) · proprietary | 97.8% | API only |
| 6 | Claude Sonnet 4.5 (Sep 29, 2025, 32K) · proprietary | 97.7% | API only |
| 7 | Qwen3 Max (Sep 23, 2025) · proprietary | 97.1% | API only |
| 8 | GPT 5 Mini (Aug 07, 2025, medium) · proprietary | 96.8% | API only |
| 9 | DeepSeek R1 0528 · 684.5B | 96.6% | ~415 GB |
| 10 | O3 Mini (Jan 31, 2025, high) · proprietary | 96.5% | API only |
| 11 | Claude Haiku 4.5 (Oct 01, 2025, 32K) · proprietary | 96.4% | API only |
| 12 | Gemini 2.5 Pro Preview (May 06) · proprietary | 95.9% | API only |
| 13 | Gemini 2.5 Pro Preview (Mar 25) · proprietary | 95.6% | API only |
| 14 | GPT 5 Nano (Aug 07, 2025, medium) · proprietary | 95.2% | API only |
| 15 | O3 Mini (Jan 31, 2025, medium) · proprietary | 95.2% | API only |
| 16 | GPT 5 Nano (Aug 07, 2025, high) · proprietary | 94.9% | API only |
| 17 | O1 (Dec 17, 2024, high) · proprietary | 94.7% | API only |
| 18 | O1 (Dec 17, 2024, medium) · proprietary | 94.4% | API only |
| 19 | DeepSeek R1 · 684.5B | 93.0% | ~415 GB |
| 20 | Claude 3.7 Sonnet (Feb 19, 2025, 64K) · proprietary | 91.2% | API only |
| 21 | Grok 3 Mini Beta (low) · proprietary | 90.9% | API only |
| 22 | Claude 3.7 Sonnet (Feb 19, 2025, 32K) · proprietary | 90.0% | API only |
| 23 | DeepSeek R1 Distill Llama 70B · 70.6B | 89.9% | ~43 GB |
| 24 | O1 Mini (Sep 12, 2024, high) · proprietary | 89.2% | API only |
| 25 | Grok 3 Beta · proprietary | 88.8% | API only |
| 26 | Grok 3 Mini Beta (high) · proprietary | 88.1% | API only |
| 27 | GPT 4.1 Mini (Apr 14, 2025) · proprietary | 87.3% | API only |
| 28 | DeepSeek R1 Distill Qwen 14B · 14.8B | 87.1% | ~9.6 GB |
| 29 | Claude Haiku 4.5 (Oct 01, 2025) · proprietary | 86.9% | API only |
| 30 | Claude 3.7 Sonnet (Feb 19, 2025, 16K) · proprietary | 86.3% | API only |
| 31 | Claude Opus 4 (May 14, 2025) · proprietary | 85.0% | API only |
| 32 | Claude Sonnet 4 (May 14, 2025) · proprietary | 84.4% | API only |
| 33 | O1 Mini (Sep 12, 2024, medium) · proprietary | 84.3% | API only |
| 34 | Gemini 2.0 Pro Exp (Feb 05) · proprietary | 83.5% | API only |
| 35 | GPT 4.1 (Apr 14, 2025) · proprietary | 83.0% | API only |
| 36 | Gemini 2.0 Flash 001 · proprietary | 82.2% | API only |
| 37 | O1 Preview (Sep 12, 2024) · proprietary | 81.7% | API only |
| 38 | Mistral Medium 2505 · proprietary | 81.6% | API only |
| 39 | GPT 4.5 Preview (Feb 27, 2025) · proprietary | 78.6% | API only |
| 40 | DeepSeek v3 0324 · 684.5B | 75.5% | ~415 GB |
| 41 | Gemma 3 27B IT · 27.4B | 74.0% | ~18 GB |
| 42 | Llama 4 Maverick 17B 128E Instruct · 401.6B | 73.0% | ~265 GB |
| 43 | Gemini 1.5 Pro 002 · proprietary | 70.4% | API only |
| 44 | GPT 4.1 Nano (Apr 14, 2025) · proprietary | 70.0% | API only |
| 45 | Qwen3 235B A22B · 235.1B | 68.9% | ~142 GB |
| 46 | Claude 3.7 Sonnet (Feb 19, 2025) · proprietary | 68.2% | API only |
| 47 | Qwen Max (Jan 25, 2025) · proprietary | 67.2% | API only |
| 48 | Qwen Plus (Jan 25, 2025) · proprietary | 65.3% | API only |
| 49 | Phi 4 · 14.7B | 64.9% | ~9.5 GB |
| 50 | DeepSeek v3 · 684.5B | 64.8% | ~415 GB |
| 51 | Grok 2 (Dec 12) · proprietary | 63.5% | API only |
| 52 | Qwen2.5 72B Instruct · 72.7B | 63.2% | ~45 GB |
| 53 | Llama 4 Scout 17B 16E Instruct · 108.6B | 62.3% | ~72 GB |
| 54 | Gemini 1.5 Flash 002 · proprietary | 61.9% | API only |
| 55 | Claude 3.5 Sonnet (Oct 22, 2024) · proprietary | 57.0% | API only |
| 56 | Qwen Turbo (Nov 01, 2024) · proprietary | 56.2% | API only |
| 57 | Qwen2.5 32B Instruct · 32.8B | 56.1% | ~21 GB |
| 58 | GPT 4o (Aug 06, 2024) · proprietary | 53.3% | API only |
| 59 | GPT 4o Mini (Jul 18, 2024) · proprietary | 52.6% | API only |
| 60 | Claude 3.5 Sonnet (Jun 20, 2024) · proprietary | 51.7% | API only |
| 61 | GPT 4o (May 13, 2024) · proprietary | 51.0% | API only |
| 62 | Mistral Large Instruct 2411 · 122.6B | 50.3% | ~75 GB |
| 63 | GPT 4o (Nov 20, 2024) · proprietary | 49.8% | API only |
| 64 | Llama 3.1 405B Instruct · 405.9B | 49.8% | ~268 GB |
| 65 | Mistral Small 3.1 24B Instruct 2503 · 24.0B | 46.8% | ~15 GB |
| 66 | GPT 4 Turbo (Apr 09, 2024) · proprietary | 46.7% | API only |
| 67 | Claude 3.5 Haiku (Oct 22, 2024) · proprietary | 46.4% | API only |
| 68 | Mistral Large Instruct 2407 · 122.6B | 44.8% | ~81 GB |
| 69 | Mistral Small 24B Instruct 2501 · 23.6B | 44.8% | ~15 GB |
| 70 | Llama 3.1 Tulu 3 70B DPO · 70.6B | 42.7% | ~43 GB |
| 71 | Llama 3.3 70B Instruct · 70.6B | 41.6% | ~47 GB |
| 72 | Gemini 1.5 Pro 001 · proprietary | 40.8% | API only |
| 73 | GPT 4 1106 Preview · proprietary | 40.0% | API only |
| 74 | Llama 3.2 90B Vision Instruct · 88.6B | 39.4% | ~195 GB |
| 75 | Qwen2 72B Instruct · 72.7B | 39.1% | ~45 GB |
| 76 | Claude 3 Opus (Feb 29, 2024) · proprietary | 37.5% | API only |
| 77 | Llama 3.1 70B Instruct · 70.6B | 36.7% | ~47 GB |
| 78 | GPT 4 0125 Preview · proprietary | 35.4% | API only |
| 79 | Gemma 2 27B IT · 27.2B | 27.9% | ~18 GB |
| 80 | WizardLM 2 8x22B · 140.6B | 25.7% | ~85 GB |
| 81 | Yi 1.5 34B Chat · 34.4B | 25.5% | ~21 GB |
| 82 | Gemini 1.5 Flash 001 · proprietary | 25.1% | API only |
| 83 | Mistral Large 2402 · proprietary | 24.5% | API only |
| 84 | Mixtral 8x22B Instruct v0.1 · 140.6B | 24.2% | ~85 GB |
| 85 | GPT 4 (Jun 13) · proprietary | 23.0% | API only |
| 86 | Llama 3.1 8B Instruct · 8.0B | 22.9% | ~5.3 GB |
| 87 | Hermes 2 Theta Llama 3 70B · 70.6B | 22.7% | ~43 GB |
| 88 | Meta Llama 3 70B Instruct · 70.6B | 22.6% | ~47 GB |
| 89 | Gemma 2 9B IT · 9.2B | 21.0% | ~6.1 GB |
| 90 | Claude 3 Sonnet (Feb 29, 2024) · proprietary | 18.2% | API only |
| 91 | Phi 3 Medium 128k Instruct · 14.0B | 17.6% | ~9.1 GB |
| 92 | GPT 3.5 Turbo (Nov 06) · proprietary | 15.9% | API only |
| 93 | Ministral 8B 2410 · proprietary | 14.9% | API only |
| 94 | Claude 3 Haiku (Mar 07, 2024) · proprietary | 14.9% | API only |
| 95 | Ministral 3B 2410 · proprietary | 14.4% | API only |
| 96 | Claude 2.0 · proprietary | 11.7% | API only |
| 97 | Dbrx Instruct · proprietary | 11.7% | API only |
| 98 | GPT 3.5 Turbo (Jan 25) · proprietary | 11.6% | API only |
| 99 | Gemini 1.0 Pro 001 · proprietary | 11.2% | API only |
| 100 | Mistral Nemo Instruct 2407 · 12.2B | 10.8% | ~8.1 GB |
| 101 | Mixtral 8x7B Instruct v0.1 · 46.7B | 10.0% | ~29 GB |
| 102 | Deepseek Llm 67B Chat · 67B | 6.4% | ~41 GB |
| 103 | Meta Llama 3 8B Instruct · 8.0B | 6.1% | ~5.3 GB |
| 104 | Yi 34B Chat · 34.4B | 5.1% | ~21 GB |
| 105 | Open Mistral 7B · proprietary | 3.7% | API only |
| 106 | Mistral 7B Instruct v0.3 · 7.2B | 3.6% | ~4.9 GB |
| 107 | Llama 2 70B Chat HF · 69.0B | 3.3% | ~152 GB |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- Mistral 7B Instruct v0.3, 7B, score 3.6% — on the efficiency frontier (best score at its size or smaller).
- Llama 3.1 8B Instruct, 8B, score 22.9% — on the efficiency frontier (best score at its size or smaller).
- Phi 4, 15B, score 64.9% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek R1 Distill Qwen 14B, 15B, score 87.1% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek R1 Distill Llama 70B, 71B, score 89.9% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek R1, 684B, score 93.0% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek R1 0528, 685B, score 96.6% — on the efficiency frontier (best score at its size or smaller).
MATH Level 5: frequently asked questions
- What is the best open LLM on MATH Level 5?
- DeepSeek R1 0528 is the top open model on MATH Level 5, scoring 96.6%. Among all models tested — including proprietary ones — it ranks #9. The top model overall is GPT 5 (Aug 07, 2025, high) (OpenAI) at 98.1%.
- What's the best MATH Level 5 model you can run on a 24 GB GPU?
- DeepSeek R1 Distill Qwen 14B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (~9.6 GB), scoring 87.1% on MATH Level 5.
- What's the best MATH Level 5 model you can run on a 12 GB GPU?
- Llama 3.1 8B Instruct is the highest-scoring open model that fits in 12 GB at 4-bit quantization (~5.3 GB), scoring 22.9% on MATH Level 5.
- Can open models match proprietary models on MATH Level 5?
- Not quite on MATH Level 5: the strongest proprietary model (GPT 5 (Aug 07, 2025, high)) scores 98.1%, ahead of the best open model (DeepSeek R1 0528) at 96.6% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.