Reasoning
DTBench Leaderboard
DTBench is a set of handcrafted, expert-validated multiple-choice questions on decision theory — Newcomb-like problems and related puzzles — built by the Conceptual Reasoning Index project with Anthropic. It tests whether a model can follow an abstract argument rather than recall a fact.
Source: epoch70 open models ranked+165 proprietaryData through Oct 2026
All models ranked on DTBench
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 | Claude Opus 5.5 Max · proprietary | 98.9% | API only |
| 2 | Claude Fable 5 Max · proprietary | 98.4% | API only |
| 3 | Claude Opus 5 (high) · proprietary | 97.9% | API only |
| 4 | Claude Opus 5 (xhigh) · proprietary | 97.9% | API only |
| 5 | Claude Fable 5.1 (unspecified) · proprietary | 97.6% | API only |
| 6 | Claude Fable 5.1 (xhigh) · proprietary | 97.6% | API only |
| 7 | Claude Opus 5 Max · proprietary | 97.6% | API only |
| 8 | Gemini 4 Argon (low) · proprietary | 97.6% | API only |
| 9 | Claude Fable 5 (xhigh) · proprietary | 97.3% | API only |
| 10 | Claude Opus 5.5 (xhigh) · proprietary | 97.3% | API only |
| 11 | Claude Sonnet 5.5 Max · proprietary | 97.3% | API only |
| 12 | Gemini 4 Argon (high) · proprietary | 97.3% | API only |
| 13 | Gemini 4 Argon (medium) · proprietary | 97.3% | API only |
| 14 | GPT 6 Astra (xhigh) · proprietary | 97.3% | API only |
| 15 | GPT 6 Sol Max · proprietary | 97.3% | API only |
| 16 | Grok 4.6 (xhigh) · proprietary | 97.3% | API only |
| 17 | Claude Fable 5.1 (medium) · proprietary | 97.1% | API only |
| 18 | Gemini 3.1 Pro Preview (medium) · proprietary | 97.1% | API only |
| 19 | GPT 6 Astra (medium) · proprietary | 97.1% | API only |
| 20 | GPT 6 Astra (unspecified) · proprietary | 97.1% | API only |
| 21 | Gemini 3.7 Flash (high) · proprietary | 96.8% | API only |
| 22 | Claude Fable 5 (high) · proprietary | 96.5% | API only |
| 23 | Claude Fable 5.1 (high) · proprietary | 96.5% | API only |
| 24 | Claude Opus 5.5 (high) · proprietary | 96.5% | API only |
| 25 | Claude Opus 5.5 (medium) · proprietary | 96.5% | API only |
| 26 | GPT 6 Astra (high) · proprietary | 96.5% | API only |
| 27 | GPT 6.1 Sol Max · proprietary | 96.5% | API only |
| 28 | Grok 4.5 (high) · proprietary | 96.5% | API only |
| 29 | Muse Spark 1.3 (unspecified) · proprietary | 96.5% | API only |
| 30 | Claude Sonnet 5.5 (xhigh) · proprietary | 96.0% | API only |
| 31 | GPT 5.5 (xhigh) · proprietary | 96.0% | API only |
| 32 | GPT 5.5 Pro (xhigh) · proprietary | 96.0% | API only |
| 33 | GPT 5.6 Sol Promax · proprietary | 96.0% | API only |
| 34 | Grok 4.7 (xhigh) · proprietary | 96.0% | API only |
| 35 | Claude Fable 5 (medium) · proprietary | 95.7% | API only |
| 36 | GPT 5.6 Sol (xhigh) · proprietary | 95.7% | API only |
| 37 | Gemini 3.8 Flash (unspecified) · proprietary | 95.7% | API only |
| 38 | Claude Opus 5.5 (low) · proprietary | 95.5% | API only |
| 39 | Gemini 3.1 Pro Preview (high) · proprietary | 95.5% | API only |
| 40 | Gemini 3.6 Flash (high) · proprietary | 95.5% | API only |
| 41 | GPT 5.6 Sol (high) · proprietary | 95.5% | API only |
| 42 | GPT 5.6 Sol Max · proprietary | 95.5% | API only |
| 43 | GPT 5.6 Sol (medium) · proprietary | 95.2% | API only |
| 44 | Muse Wonder 1.0 (unspecified) · proprietary | 95.2% | API only |
| 45 | Claude Haiku 5.5 Max · proprietary | 95.0% | API only |
| 46 | Claude Opus 4.8 Max · proprietary | 94.9% | API only |
| 47 | Claude Opus 5 (medium) · proprietary | 94.9% | API only |
| 48 | Gemini 3.1 Pro Preview (low) · proprietary | 94.9% | API only |
| 49 | Gemini 3.8 Flash (low) · proprietary | 94.9% | API only |
| 50 | Gemini 3.8 Flash (medium) · proprietary | 94.9% | API only |
| 51 | GPT 6 Astra (low) · proprietary | 94.9% | API only |
| 52 | Gemini 3.5 Flash (high) · proprietary | 94.7% | API only |
| 53 | Gemini 3.7 Flash (medium) · proprietary | 94.7% | API only |
| 54 | Muse Spark 1.2 (xhigh) · proprietary | 94.7% | API only |
| 55 | Claude Opus 4.7 Max · proprietary | 94.7% | API only |
| 56 | GPT 5.4 (Mar 05, 2026, xhigh) · proprietary | 94.4% | API only |
| 57 | Muse Spark 1.1 (high) · proprietary | 94.4% | API only |
| 58 | Claude Fable 5.1 (low) · proprietary | 94.1% | API only |
| 59 | Claude Sonnet 5.5 (high) · proprietary | 94.1% | API only |
| 60 | Claude Sonnet 5.5 (medium) · proprietary | 93.9% | API only |
| 61 | DeepSeek V4 Pro 0813 · 1650.5B | 93.9% | ~991 GB |
| 62 | GPT 5.6 Sol (low) · proprietary | 93.9% | API only |
| 63 | GLM 5.2 · 753.3B | 93.6% | ~456 GB |
| 64 | Claude Sonnet 5.5 (low) · proprietary | 93.3% | API only |
| 65 | Gemini 3.7 Flash (low) · proprietary | 93.3% | API only |
| 66 | GPT 5.6 Terra Max · proprietary | 93.3% | API only |
| 67 | Claude Opus 5 (low) · proprietary | 93.1% | API only |
| 68 | GPT 5.6 Terra (xhigh) · proprietary | 92.5% | API only |
| 69 | Claude Sonnet 5 Max · proprietary | 92.5% | API only |
| 70 | Qwen3.7 Max · proprietary | 92.3% | API only |
| 71 | Qwen3.8 Max (xhigh) · proprietary | 92.0% | API only |
| 72 | Claude Fable 5 (low) · proprietary | 91.5% | API only |
| 73 | Claude Opus 4.6 Max · proprietary | 91.2% | API only |
| 74 | GPT 5.6 Terra (high) · proprietary | 91.2% | API only |
| 75 | Kimi K3 · 2779.9B | 91.2% | ~1674 GB |
| 76 | DeepSeek V4 Flash 0731 · 304.2B | 90.9% | ~183 GB |
| 77 | GPT 5.2 (Dec 11, 2025, xhigh) · proprietary | 90.9% | API only |
| 78 | Kimi K2.6 · 1026.9B | 90.9% | ~620 GB |
| 79 | DeepSeek V4 Pro · 1598.8B | 90.7% | ~960 GB |
| 80 | GPT 5 (Aug 07, 2025, high) · proprietary | 90.7% | API only |
| 81 | Grok 4.3 (high) · proprietary | 90.7% | API only |
| 82 | GPT 5.1 (Nov 13, 2025, high) · proprietary | 90.1% | API only |
| 83 | GPT 5.6 Terra (low) · proprietary | 90.1% | API only |
| 84 | GPT 5.6 Terra (medium) · proprietary | 90.1% | API only |
| 85 | GPT 6 Luna Max · proprietary | 90.1% | API only |
| 86 | Grok 4.20 · proprietary | 90.1% | API only |
| 87 | NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B | 90.1% | ~370 GB |
| 88 | Claude Opus 4.5 (Nov 01, 2025, high) · proprietary | 89.9% | API only |
| 89 | Claude Sonnet 4.6 Max · proprietary | 89.9% | API only |
| 90 | DeepSeek V4.1 Flash · 763.2B | 89.9% | ~458 GB |
| 91 | Claude Sonnet 5 (xhigh) · proprietary | 89.5% | API only |
| 92 | Gemini 3 Flash Preview (high) · proprietary | 89.1% | API only |
| 93 | GPT 5.6 Luna (xhigh) · proprietary | 89.1% | API only |
| 94 | GPT 5.6 Luna Max · proprietary | 88.8% | API only |
| 95 | Mistral Large 4 1T A52B · 1000B | 88.2% | ~604 GB |
| 96 | Qwen3.8 27B · 27.8B | 88.0% | ~17 GB |
| 97 | DeepSeek V3.2 Exp · 685.4B | 87.7% | ~415 GB |
| 98 | Grok 4.1 Fast Reasoning · proprietary | 87.7% | API only |
| 99 | GLM 5.3 · 753.3B | 87.7% | ~456 GB |
| 100 | Inkling · 952.4B | 87.5% | ~572 GB |
| 101 | Qwen3.5 397B A17B · 403.4B | 87.5% | ~242 GB |
| 102 | Claude Haiku 5.5 (xhigh) · proprietary | 87.2% | API only |
| 103 | Qwen3.6 Max Preview · proprietary | 87.2% | API only |
| 104 | O3 Pro (Jun 10, 2025, high) · proprietary | 86.9% | API only |
| 105 | Claude Haiku 5.5 (high) · proprietary | 86.7% | API only |
| 106 | GPT 5.6 Luna (high) · proprietary | 86.7% | API only |
| 107 | DeepSeek V4 Flash · 290.9B | 86.4% | ~175 GB |
| 108 | DeepSeek V3.2 · 685.4B | 85.6% | ~415 GB |
| 109 | O3 (Apr 16, 2025, high) · proprietary | 84.8% | API only |
| 110 | Claude Sonnet 5 (high) · proprietary | 84.5% | API only |
| 111 | MiMo V2.5 Pro · 1023.2B | 84.5% | ~614 GB |
| 112 | Claude Haiku 5.5 (medium) · proprietary | 84.3% | API only |
| 113 | Qwen3.5 122B A10B · 125.1B | 84.3% | ~75 GB |
| 114 | Qwen3.7 Plus · proprietary | 84.0% | API only |
| 115 | GPT 5.6 Sol None · proprietary | 83.7% | API only |
| 116 | Gemini 3.5 Flash Lite (high) · proprietary | 83.5% | API only |
| 117 | Claude Sonnet 4.5 (Sep 29, 2025, unspecified) · proprietary | 83.2% | API only |
| 118 | GPT 5.6 Luna (medium) · proprietary | 83.2% | API only |
| 119 | Qwen3.5 Flash · proprietary | 82.9% | API only |
| 120 | DeepSeek V3.1 · 684.5B | 82.7% | ~415 GB |
| 121 | Gemma 4 31B IT · 31.3B | 82.7% | ~20 GB |
| 122 | Grok 4 Fast · proprietary | 82.7% | API only |
| 123 | Gemini 2.5 Pro · proprietary | 82.4% | API only |
| 124 | Qwen3.5 27B · 27.8B | 82.4% | ~17 GB |
| 125 | Qwen3 Max (Sep 23, 2025) · proprietary | 82.1% | API only |
| 126 | Qwen3.6 Plus · proprietary | 81.9% | API only |
| 127 | Claude Opus 4 (May 14, 2025, unspecified) · proprietary | 81.6% | API only |
| 128 | DeepSeek V3.1 Terminus · 684.5B | 81.3% | ~415 GB |
| 129 | Claude Haiku 5.5 (low) · proprietary | 81.1% | API only |
| 130 | Qwen Plus (Apr 28, 2025) · proprietary | 81.1% | API only |
| 131 | GPT 5 Mini (Aug 07, 2025, high) · proprietary | 80.5% | API only |
| 132 | Qwen3.5 Plus · proprietary | 80.5% | API only |
| 133 | GPT 5.4 Nano (Mar 17, 2026, xhigh) · proprietary | 80.3% | API only |
| 134 | Qwen3 235B A22B Thinking 2507 · 235.1B | 80.3% | ~142 GB |
| 135 | Claude Opus 4.1 (Aug 05, 2025, unspecified) · proprietary | 80.0% | API only |
| 136 | GPT 5.4 Mini (Mar 17, 2026, xhigh) · proprietary | 80.0% | API only |
| 137 | GPT 5.6 Luna (low) · proprietary | 80.0% | API only |
| 138 | Qwen3.5 35B A3B · 36.0B | 80.0% | ~22 GB |
| 139 | Claude Sonnet 5 (medium) · proprietary | 79.2% | API only |
| 140 | MiniMax M3 · 427.0B | 78.9% | ~257 GB |
| 141 | GPT 5.6 Terra None · proprietary | 78.7% | API only |
| 142 | Qwen3 235B A22B Instruct 2507 · 235.1B | 78.4% | ~142 GB |
| 143 | Qwen3.6 27B · 27.8B | 78.1% | ~17 GB |
| 144 | O4 Mini (Apr 16, 2025, high) · proprietary | 77.6% | API only |
| 145 | Claude Sonnet 4 (May 14, 2025, unspecified) · proprietary | 77.1% | API only |
| 146 | Qwen3.6 Flash · proprietary | 77.1% | API only |
| 147 | Gemini 3.1 Flash Lite (high) · proprietary | 76.8% | API only |
| 148 | Gemini 2.5 Flash · proprietary | 76.5% | API only |
| 149 | GPT OSS 120B · 116.8B | 76.3% | ~71 GB |
| 150 | Qwen3 235B A22B · 235.1B | 75.7% | ~142 GB |
| 151 | Mistral Medium 3.5 128B · 127.7B | 75.5% | ~78 GB |
| 152 | Claude Haiku 5.5 None · proprietary | 75.2% | API only |
| 153 | Gemma 4 26B A4B IT · 25.8B | 74.9% | ~16 GB |
| 154 | O1 (Dec 17, 2024, high) · proprietary | 74.7% | API only |
| 155 | Qwen3.6 35B A3B · 36.0B | 73.9% | ~22 GB |
| 156 | Claude Haiku 4.5 (Oct 01, 2025, unspecified) · proprietary | 73.6% | API only |
| 157 | Claude Sonnet 5 (low) · proprietary | 73.6% | API only |
| 158 | Qwen3.5 9B · 9.7B | 71.2% | ~6.4 GB |
| 159 | Mistral Small 4 119B 2603 · 119.4B | 70.9% | ~73 GB |
| 160 | Qwen3 30B A3B Thinking 2507 · 30.5B | 69.3% | ~19 GB |
| 161 | GPT 5.6 Luna None · proprietary | 69.3% | API only |
| 162 | GPT 4.1 Mini (Apr 14, 2025) · proprietary | 68.8% | API only |
| 163 | O3 Mini (Jan 31, 2025, high) · proprietary | 68.8% | API only |
| 164 | GPT 4.1 (Apr 14, 2025) · proprietary | 68.3% | API only |
| 165 | GPT OSS 20B · 20.9B | 68.0% | ~13 GB |
| 166 | Claude 3.5 Sonnet (Oct 22, 2024) · proprietary | 67.8% | API only |
| 167 | Claude 3.5 Sonnet (Jun 20, 2024) · proprietary | 67.8% | API only |
| 168 | Mistral Medium 2508 · proprietary | 67.5% | API only |
| 169 | Qwen3 32B · 32.8B | 67.5% | ~20 GB |
| 170 | Qwen3 30B A3B Instruct 2507 · 30.5B | 67.2% | ~19 GB |
| 171 | Grok 2 (Dec 12) · proprietary | 65.2% | API only |
| 172 | Mistral Large 3 675B Instruct 2512 · 675B | 65.1% | ~446 GB |
| 173 | DeepSeek v3 0324 · 684.5B | 64.8% | ~415 GB |
| 174 | GPT 4o (May 13, 2024) · proprietary | 64.5% | API only |
| 175 | Qwen3 14B · 14.8B | 64.0% | ~9.5 GB |
| 176 | Gemini 2.0 Flash 001 · proprietary | 63.2% | API only |
| 177 | Qwen2.5 72B Instruct · 72.7B | 62.9% | ~45 GB |
| 178 | Gemini 2.5 Flash Lite Preview Thinking (Jun 17) · proprietary | 62.8% | API only |
| 179 | DeepSeek Chat · proprietary | 62.7% | API only |
| 180 | GPT 4 (Jun 13) · proprietary | 62.7% | API only |
| 181 | GPT 5 Nano (Aug 07, 2025, high) · proprietary | 62.7% | API only |
| 182 | Mistral Medium 2505 · proprietary | 62.3% | API only |
| 183 | Llama 4 Maverick 17B 128E Instruct · 401.6B | 61.9% | ~265 GB |
| 184 | Claude 3 Opus (Feb 29, 2024) · proprietary | 61.6% | API only |
| 185 | GPT 4 Turbo · proprietary | 61.6% | API only |
| 186 | Llama 3.1 405B Instruct · 405.9B | 61.4% | ~268 GB |
| 187 | C4ai Command A 03 2025 · 111.1B | 61.3% | ~73 GB |
| 188 | Magistral Small 2509 · 24.0B | 61.3% | ~15 GB |
| 189 | Mistral Large Instruct 2407 · 122.6B | 61.2% | ~81 GB |
| 190 | Mistral Large Instruct 2411 · 122.6B | 60.8% | ~75 GB |
| 191 | Qwen3 30B A3B · 30.5B | 60.3% | ~19 GB |
| 192 | Llama 3.1 70B Instruct · 70.6B | 60.0% | ~47 GB |
| 193 | Mistral Small 3.2 24B Instruct 2506 · 24.0B | 59.9% | ~15 GB |
| 194 | Qwen3 8B · 8.2B | 59.7% | ~5.5 GB |
| 195 | Llama 3.3 70B Instruct · 70.6B | 59.5% | ~47 GB |
| 196 | Gemini 1.5 Pro 002 · proprietary | 59.0% | API only |
| 197 | Mistral Small 3.1 24B Instruct 2503 · 24.0B | 58.6% | ~15 GB |
| 198 | Llama 4 Scout 17B 16E Instruct · 108.6B | 57.9% | ~72 GB |
| 199 | Claude 3.5 Haiku (Oct 22, 2024) · proprietary | 56.7% | API only |
| 200 | Gemini 1.5 Pro 001 · proprietary | 56.5% | API only |
| 201 | Mistral Large 2402 · proprietary | 55.9% | API only |
| 202 | Mixtral 8x22B Instruct v0.1 · 140.6B | 55.1% | ~85 GB |
| 203 | C4ai Command R Plus 08 2024 · 103.8B | 54.9% | ~69 GB |
| 204 | GPT 4o Mini (Jul 18, 2024) · proprietary | 54.4% | API only |
| 205 | Meta Llama 3 70B Instruct · 70.6B | 54.2% | ~47 GB |
| 206 | Gemini 1.5 Flash 002 · proprietary | 53.8% | API only |
| 207 | Claude 3 Sonnet (Feb 29, 2024) · proprietary | 53.6% | API only |
| 208 | Mistral Medium 2312 · proprietary | 53.3% | API only |
| 209 | Mistral Small 2402 · proprietary | 53.0% | API only |
| 210 | Gemini 2.0 Flash Lite · proprietary | 52.5% | API only |
| 211 | Gemma 3 27B IT · 27.4B | 52.5% | ~18 GB |
| 212 | GPT 4.1 Nano (Apr 14, 2025) · proprietary | 52.5% | API only |
| 213 | Claude 2.0 · proprietary | 51.8% | API only |
| 214 | Ministral 3B 2410 · proprietary | 51.7% | API only |
| 215 | Gemini 1.5 Flash 001 · proprietary | 51.4% | API only |
| 216 | Claude 2.1 · proprietary | 50.9% | API only |
| 217 | Gemma 3 4B IT · 4.3B | 50.9% | ~2.8 GB |
| 218 | Llama 3.1 8B Instruct · 8.0B | 50.9% | ~5.3 GB |
| 219 | Claude 3 Haiku (Mar 07, 2024) · proprietary | 50.1% | API only |
| 220 | Gemini 1.5 Flash 8B 001 · proprietary | 50.0% | API only |
| 221 | Mixtral 8x7B Instruct v0.1 · 46.7B | 49.6% | ~29 GB |
| 222 | Mistral Small Instruct 2409 · 22.2B | 49.5% | ~14 GB |
| 223 | Gemma 3 12B IT · 12.2B | 48.8% | ~8.0 GB |
| 224 | Mistral Nemo Instruct 2407 · 12.2B | 48.6% | ~8.1 GB |
| 225 | GPT 3.5 Turbo (Jan 25) · proprietary | 48.5% | API only |
| 226 | Gemma 2 27B · proprietary | 48.0% | API only |
| 227 | Qwen2.5 7B Instruct · 7.6B | 47.7% | ~5.0 GB |
| 228 | C4ai Command R 08 2024 · 32.3B | 46.4% | ~21 GB |
| 229 | Gemini 1.0 Pro 001 · proprietary | 45.9% | API only |
| 230 | Claude Instant 1.2 · proprietary | 45.8% | API only |
| 231 | Ministral 8B 2410 · proprietary | 45.7% | API only |
| 232 | Meta Llama 3 8B Instruct · 8.0B | 43.9% | ~5.3 GB |
| 233 | Open Mistral 7B · proprietary | 42.5% | API only |
| 234 | Llama 2 13B Chat HF · 13.0B | 42.2% | ~8.6 GB |
| 235 | Llama 2 70B Chat HF · 69.0B | 41.6% | ~152 GB |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- Gemma 3 4B IT, 4B, score 50.9% — on the efficiency frontier (best score at its size or smaller).
- Qwen3 8B, 8B, score 59.7% — on the efficiency frontier (best score at its size or smaller).
- Qwen3.5 9B, 10B, score 71.2% — on the efficiency frontier (best score at its size or smaller).
- Gemma 4 26B A4B IT, 26B, score 74.9% — on the efficiency frontier (best score at its size or smaller).
- Qwen3.8 27B, 28B, score 88.0% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Flash 0731, 304B, score 90.9% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.2, 753B, score 93.6% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Pro 0813, 1.7T, score 93.9% — on the efficiency frontier (best score at its size or smaller).
DTBench: frequently asked questions
- What is the best open LLM on DTBench?
- DeepSeek V4 Pro 0813 is the top open model on DTBench, scoring 93.9%. Among all models tested — including proprietary ones — it ranks #60. The top model overall is Claude Opus 5.5 Max (Anthropic) at 98.9%.
- What's the best DTBench model you can run on a 24 GB GPU?
- Qwen3.8 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (~17 GB), scoring 88.0% on DTBench.
- What's the best DTBench model you can run on a 12 GB GPU?
- Qwen3.5 9B is the highest-scoring open model that fits in 12 GB at 4-bit quantization (~6.4 GB), scoring 71.2% on DTBench.
- Can open models match proprietary models on DTBench?
- Not quite on DTBench: the strongest proprietary model (Claude Opus 5.5 Max) scores 98.9%, ahead of the best open model (DeepSeek V4 Pro 0813) at 93.9% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.