Reasoning
GPQA Diamond Leaderboard
GPQA Diamond is a set of extremely hard, graduate-level science questions (physics, chemistry, biology) written by domain experts and filtered so that skilled non-experts with web access still fail. It measures genuine reasoning rather than memorization.
Source: epoch98 open models ranked+202 proprietaryData through Oct 2026
Open models ranked on GPQA Diamond
# shows rank among open models / rank overall (including proprietary).
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 / 21 | Kimi K3 · 2779.9B | 93.1% | ~1674 GB |
| 2 / 28 | GLM 5.2 · 753.3B | 91.9% | ~456 GB |
| 3 / 29 | DeepSeek V4 Pro 0813 · 1650.5B | 91.7% | ~991 GB |
| 4 / 33 | DeepSeek V4 Flash 0731 · 304.2B | 91.0% | ~183 GB |
| 5 / 34 | DeepSeek V4 Pro · 1598.8B | 90.9% | ~960 GB |
| 6 / 35 | GLM 5.3 · 753.3B | 90.9% | ~456 GB |
| 7 / 36 | MiniMax M3 · 427.0B | 90.9% | ~257 GB |
| 8 / 38 | Kimi K2.6 · 1026.9B | 90.8% | ~620 GB |
| 9 / 45 | GLM 5.3 Flash · 321.3B | 90.1% | ~195 GB |
| 10 / 46 | GLM 5.1 · 753.9B | 89.9% | ~457 GB |
| 11 / 49 | DeepSeek V4.1 Flash · 763.2B | 89.8% | ~458 GB |
| 12 / 58 | Muse Spark Open 816B A42B · 822.0B | 88.6% | ~543 GB |
| 13 / 59 | Inkling Small · 266.0B | 88.5% | ~160 GB |
| 14 / 63 | Inkling · 952.4B | 88.3% | ~572 GB |
| 15 / 68 | Kimi K2.7 Code · 1026.9B | 87.9% | ~620 GB |
| 16 / 70 | GLM 5 · 753.9B | 87.8% | ~457 GB |
| 17 / 72 | Kimi K2.5 · 1026.9B | 87.6% | ~620 GB |
| 18 / 81 | Qwen3.5 397B A17B · 403.4B | 86.4% | ~242 GB |
| 19 / 86 | Qwen3.6 27B · 27.8B | 85.9% | ~17 GB |
| 20 / 90 | NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B | 85.4% | ~370 GB |
| 21 / 96 | Qwen3.6 35B A3B · 36.0B | 84.9% | ~22 GB |
| 22 / 97 | Kimi K2 Thinking · 1026.4B | 84.2% | ~620 GB |
| 23 / 100 | Qwen3.5 35B A3B · 36.0B | 83.5% | ~22 GB |
| 24 / 106 | GLM 4.7 · 358.3B | 83.3% | ~216 GB |
| 25 / 124 | Qwen3 235B A22B Thinking 2507 · 235.1B | 80.0% | ~142 GB |
| 26 / 127 | Qwen3.5 9B · 9.7B | 79.0% | ~6.4 GB |
| 27 / 144 | DeepSeek R1 0528 · 684.5B | 76.3% | ~415 GB |
| 28 / 149 | Gemma 4 31B IT · 31.3B | 75.8% | ~20 GB |
| 29 / 150 | GPT OSS 120B · 116.8B | 75.8% | ~71 GB |
| 30 / 163 | Gemma 4 26B A4B IT · 25.8B | 73.2% | ~16 GB |
| 31 / 170 | Seed OSS 36B Instruct · 36.2B | 71.5% | ~22 GB |
| 32 / 173 | Qwen3 235B A22B · 235.1B | 70.7% | ~142 GB |
| 33 / 174 | Qwen3 30B A3B Thinking 2507 · 30.5B | 70.1% | ~19 GB |
| 34 / 176 | DeepSeek R1 · 684.5B | 69.2% | ~415 GB |
| 35 / 180 | DeepSeek v3 0324 · 684.5B | 67.6% | ~415 GB |
| 36 / 183 | Llama 4 Maverick 17B 128E Instruct · 401.6B | 67.0% | ~265 GB |
| 37 / 190 | Qwen3 32B · 32.8B | 65.7% | ~20 GB |
| 38 / 193 | QwQ 32B · 32.8B | 65.3% | ~21 GB |
| 39 / 194 | DeepSeek R1 Distill Qwen 32B · 32.8B | 64.1% | ~21 GB |
| 40 / 197 | Qwen3 14B · 14.8B | 63.8% | ~9.5 GB |
| 41 / 200 | Qwen3 30B A3B · 30.5B | 61.7% | ~19 GB |
| 42 / 201 | GPT OSS 20B · 20.9B | 60.8% | ~13 GB |
| 43 / 202 | GLM 4.7 Flash · 31.2B | 60.5% | ~20 GB |
| 44 / 209 | Qwen3 8B · 8.2B | 56.8% | ~5.5 GB |
| 45 / 210 | DeepSeek v3 · 684.5B | 56.5% | ~415 GB |
| 46 / 212 | Magistral Small 2506 · 23.6B | 56.1% | ~15 GB |
| 47 / 213 | Phi 4 · 14.7B | 56.1% | ~9.5 GB |
| 48 / 214 | DeepSeek R1 Distill Llama 70B · 70.6B | 55.7% | ~43 GB |
| 49 / 215 | Qwen3 30B A3B Instruct 2507 · 30.5B | 55.6% | ~19 GB |
| 50 / 220 | Qwen3 4B · 4.0B | 52.3% | ~2.9 GB |
| 51 / 221 | Llama 4 Scout 17B 16E Instruct · 108.6B | 51.8% | ~72 GB |
| 52 / 222 | Mistral Large Instruct 2411 · 122.6B | 51.3% | ~75 GB |
| 53 / 223 | Llama 3.1 405B Instruct · 405.9B | 50.9% | ~268 GB |
| 54 / 226 | Qwen2.5 72B Instruct · 72.7B | 49.1% | ~45 GB |
| 55 / 227 | Mistral Small 3.2 24B Instruct 2506 · 24.0B | 49.0% | ~15 GB |
| 56 / 228 | Mistral Large Instruct 2407 · 122.6B | 49.0% | ~81 GB |
| 57 / 234 | Gemma 3 27B IT · 27.4B | 47.7% | ~18 GB |
| 58 / 235 | Magistral Small 2509 · 24.0B | 47.6% | ~15 GB |
| 59 / 236 | Mistral Small 3.1 24B Instruct 2503 · 24.0B | 47.5% | ~15 GB |
| 60 / 237 | Llama 3.3 70B Instruct · 70.6B | 47.4% | ~47 GB |
| 61 / 239 | Mistral Small 24B Instruct 2501 · 23.6B | 47.3% | ~15 GB |
| 62 / 242 | Llama 3.1 Tulu 3 70B DPO · 70.6B | 46.3% | ~43 GB |
| 63 / 243 | Qwen2.5 32B Instruct · 32.8B | 46.1% | ~21 GB |
| 64 / 245 | Qwen3 4B Instruct 2507 · 4.0B | 45.8% | ~2.9 GB |
| 65 / 246 | DeepSeek R1 Distill Qwen 14B · 14.8B | 44.7% | ~9.6 GB |
| 66 / 247 | Llama 3.1 70B Instruct · 70.6B | 44.2% | ~47 GB |
| 67 / 248 | WizardLM 2 8x22B · 140.6B | 43.4% | ~85 GB |
| 68 / 252 | Llama 3.2 90B Vision Instruct · 88.6B | 41.0% | ~195 GB |
| 69 / 253 | Qwen2 72B Instruct · 72.7B | 40.8% | ~45 GB |
| 70 / 255 | Meta Llama 3 70B Instruct · 70.6B | 40.6% | ~47 GB |
| 71 / 257 | Gemma 3 12B IT · 12.2B | 39.5% | ~8.0 GB |
| 72 / 260 | Qwen3 1.7B · 2.0B | 38.0% | ~1.8 GB |
| 73 / 262 | Hermes 2 Theta Llama 3 70B · 70.6B | 37.5% | ~43 GB |
| 74 / 263 | Gemma 2 27B IT · 27.2B | 36.5% | ~18 GB |
| 75 / 266 | Qwen2.5 7B Instruct · 7.6B | 35.5% | ~5.0 GB |
| 76 / 268 | Mixtral 8x22B Instruct v0.1 · 140.6B | 34.1% | ~85 GB |
| 77 / 270 | Eurus 2 7B PRIME · 7.6B | 33.9% | ~5.0 GB |
| 78 / 271 | DeepSeek R1 Distill Qwen 1.5B · 1.8B | 33.6% | ~1.4 GB |
| 79 / 275 | Yi 1.5 34B Chat · 34.4B | 32.0% | ~21 GB |
| 80 / 276 | Qwen1.5 32B Chat · 32.5B | 30.7% | ~20 GB |
| 81 / 278 | Mixtral 8x7B Instruct v0.1 · 46.7B | 30.6% | ~29 GB |
| 82 / 279 | Mistral Nemo Instruct 2407 · 12.2B | 29.9% | ~8.1 GB |
| 83 / 280 | Qwen1.5 72B Chat · 72.3B | 28.8% | ~49 GB |
| 84 / 281 | Granite 4.0 Micro · 3.4B | 28.3% | ~2.5 GB |
| 85 / 283 | Phi 3 Medium 128k Instruct · 14.0B | 27.6% | ~9.1 GB |
| 86 / 284 | Gemma 2 9B IT · 9.2B | 27.5% | ~6.1 GB |
| 87 / 287 | Llama 3.1 8B Instruct · 8.0B | 27.0% | ~5.3 GB |
| 88 / 288 | Llama 2 70B Chat HF · 69.0B | 26.3% | ~152 GB |
| 89 / 289 | Meta Llama 3 8B Instruct · 8.0B | 26.1% | ~5.3 GB |
| 90 / 291 | Deepseek Llm 67B Chat · 67B | 24.6% | ~41 GB |
| 91 / 292 | Granite 4.0 1B · 1.6B | 24.0% | ~1.4 GB |
| 92 / 293 | Llama 3.2 1B Instruct · 1.2B | 23.9% | ~0.8 GB |
| 93 / 294 | Gemma 3 4B IT · 4.3B | 23.2% | ~2.8 GB |
| 94 / 295 | Gemma 3 1B IT · 1000M | 20.0% | ~0.7 GB |
| 95 / 296 | Mistral 7B Instruct v0.3 · 7.2B | 15.2% | ~4.9 GB |
| 96 / 297 | Yi 34B Chat · 34.4B | 14.7% | ~21 GB |
| 97 / 299 | Granite 4.0 350M · 352M | 11.2% | ~0.6 GB |
| 98 / 300 | DeepSeek R1 0528 Qwen3 8B · 8.2B | 9.3% | ~5.5 GB |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- Granite 4.0 350M, 352M, score 11.2% — on the efficiency frontier (best score at its size or smaller).
- Gemma 3 1B IT, 1000M, score 20.0% — on the efficiency frontier (best score at its size or smaller).
- Llama 3.2 1B Instruct, 1B, score 23.9% — on the efficiency frontier (best score at its size or smaller).
- Granite 4.0 1B, 2B, score 24.0% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek R1 Distill Qwen 1.5B, 2B, score 33.6% — on the efficiency frontier (best score at its size or smaller).
- Qwen3 1.7B, 2B, score 38.0% — on the efficiency frontier (best score at its size or smaller).
- Qwen3 4B, 4B, score 52.3% — on the efficiency frontier (best score at its size or smaller).
- Qwen3 8B, 8B, score 56.8% — on the efficiency frontier (best score at its size or smaller).
- Qwen3.5 9B, 10B, score 79.0% — on the efficiency frontier (best score at its size or smaller).
- Qwen3.6 27B, 28B, score 85.9% — on the efficiency frontier (best score at its size or smaller).
- Inkling Small, 266B, score 88.5% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Flash 0731, 304B, score 91.0% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.2, 753B, score 91.9% — on the efficiency frontier (best score at its size or smaller).
- Kimi K3, 2.8T, score 93.1% — on the efficiency frontier (best score at its size or smaller).
GPQA Diamond: frequently asked questions
- What is the best open LLM on GPQA Diamond?
- Kimi K3 is the top open model on GPQA Diamond, scoring 93.1%. Among all models tested — including proprietary ones — it ranks #21. The top model overall is Gemini 4 Argon (high) (Google DeepMind) at 96.4%.
- What's the best GPQA Diamond model you can run on a 24 GB GPU?
- Qwen3.6 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (~17 GB), scoring 85.9% on GPQA Diamond.
- What's the best GPQA Diamond model you can run on a 12 GB GPU?
- Qwen3.5 9B is the highest-scoring open model that fits in 12 GB at 4-bit quantization (~6.4 GB), scoring 79.0% on GPQA Diamond.
- Can open models match proprietary models on GPQA Diamond?
- Not quite on GPQA Diamond: the strongest proprietary model (Gemini 4 Argon (high)) scores 96.4%, ahead of the best open model (Kimi K3) at 93.1% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.