Reasoning
DTBench Leaderboard
DTBench is a set of handcrafted, expert-validated multiple-choice questions on decision theory — Newcomb-like problems and related puzzles — built by the Conceptual Reasoning Index project with Anthropic. It tests whether a model can follow an abstract argument rather than recall a fact.
Source: epoch67 open models ranked+143 proprietaryData through Sep 2026
All models ranked on DTBench
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score |
|---|---|---|
| 1 | Claude Fable 5 Max · proprietary | 98.4% |
| 2 | Claude Opus 5 (high) · proprietary | 97.9% |
| 3 | Claude Opus 5 (xhigh) · proprietary | 97.9% |
| 4 | Claude Fable 5.1 (unspecified) · proprietary | 97.6% |
| 5 | Claude Fable 5.1 (xhigh) · proprietary | 97.6% |
| 6 | Claude Opus 5 Max · proprietary | 97.6% |
| 7 | Claude Fable 5 (xhigh) · proprietary | 97.3% |
| 8 | GPT 6 Astra (xhigh) · proprietary | 97.3% |
| 9 | Grok 4.6 (xhigh) · proprietary | 97.3% |
| 10 | Claude Fable 5.1 (medium) · proprietary | 97.1% |
| 11 | Gemini 3.1 Pro Preview (medium) · proprietary | 97.1% |
| 12 | GPT 6 Astra (medium) · proprietary | 97.1% |
| 13 | GPT 6 Astra (unspecified) · proprietary | 97.1% |
| 14 | Gemini 3.7 Flash (high) · proprietary | 96.8% |
| 15 | Claude Fable 5 (high) · proprietary | 96.5% |
| 16 | Claude Fable 5.1 (high) · proprietary | 96.5% |
| 17 | GPT 6 Astra (high) · proprietary | 96.5% |
| 18 | Grok 4.5 (high) · proprietary | 96.5% |
| 19 | Muse Spark 1.3 (unspecified) · proprietary | 96.5% |
| 20 | GPT 5.5 (xhigh) · proprietary | 96.0% |
| 21 | GPT 5.5 Pro (xhigh) · proprietary | 96.0% |
| 22 | GPT 5.6 Sol Promax · proprietary | 96.0% |
| 23 | Claude Fable 5 (medium) · proprietary | 95.7% |
| 24 | GPT 5.6 Sol (xhigh) · proprietary | 95.7% |
| 25 | Gemini 3.8 Flash (unspecified) · proprietary | 95.7% |
| 26 | Gemini 3.1 Pro Preview (high) · proprietary | 95.5% |
| 27 | Gemini 3.6 Flash (high) · proprietary | 95.5% |
| 28 | GPT 5.6 Sol (high) · proprietary | 95.5% |
| 29 | GPT 5.6 Sol Max · proprietary | 95.5% |
| 30 | GPT 5.6 Sol (medium) · proprietary | 95.2% |
| 31 | Claude Opus 4.8 Max · proprietary | 94.9% |
| 32 | Claude Opus 5 (medium) · proprietary | 94.9% |
| 33 | Gemini 3.1 Pro Preview (low) · proprietary | 94.9% |
| 34 | Gemini 3.8 Flash (low) · proprietary | 94.9% |
| 35 | Gemini 3.8 Flash (medium) · proprietary | 94.9% |
| 36 | GPT 6 Astra (low) · proprietary | 94.9% |
| 37 | Gemini 3.5 Flash (high) · proprietary | 94.7% |
| 38 | Gemini 3.7 Flash (medium) · proprietary | 94.7% |
| 39 | Muse Spark 1.2 (xhigh) · proprietary | 94.7% |
| 40 | Claude Opus 4.7 Max · proprietary | 94.7% |
| 41 | GPT 5.4 (Mar 05, 2026, xhigh) · proprietary | 94.4% |
| 42 | Muse Spark 1.1 (high) · proprietary | 94.4% |
| 43 | Claude Fable 5.1 (low) · proprietary | 94.1% |
| 44 | DeepSeek V4 Pro 0813 · 1650.5B | 93.9% |
| 45 | GPT 5.6 Sol (low) · proprietary | 93.9% |
| 46 | GLM 5.2 · 753.3B | 93.6% |
| 47 | Gemini 3.7 Flash (low) · proprietary | 93.3% |
| 48 | GPT 5.6 Terra Max · proprietary | 93.3% |
| 49 | Claude Opus 5 (low) · proprietary | 93.1% |
| 50 | GPT 5.6 Terra (xhigh) · proprietary | 92.5% |
| 51 | Claude Sonnet 5 Max · proprietary | 92.5% |
| 52 | Qwen3.7 Max · proprietary | 92.3% |
| 53 | Qwen3.8 Max (xhigh) · proprietary | 92.0% |
| 54 | Claude Fable 5 (low) · proprietary | 91.5% |
| 55 | Claude Opus 4.6 Max · proprietary | 91.2% |
| 56 | GPT 5.6 Terra (high) · proprietary | 91.2% |
| 57 | Kimi K3 · 2779.9B | 91.2% |
| 58 | DeepSeek V4 Flash 0731 · 304.2B | 90.9% |
| 59 | GPT 5.2 (Dec 11, 2025, xhigh) · proprietary | 90.9% |
| 60 | Kimi K2.6 · 1026.9B | 90.9% |
| 61 | DeepSeek V4 Pro · 1598.8B | 90.7% |
| 62 | GPT 5 (Aug 07, 2025, high) · proprietary | 90.7% |
| 63 | Grok 4.3 (high) · proprietary | 90.7% |
| 64 | GPT 5.1 (Nov 13, 2025, high) · proprietary | 90.1% |
| 65 | GPT 5.6 Terra (low) · proprietary | 90.1% |
| 66 | GPT 5.6 Terra (medium) · proprietary | 90.1% |
| 67 | Grok 4.20 · proprietary | 90.1% |
| 68 | NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B | 90.1% |
| 69 | Claude Opus 4.5 (Nov 01, 2025, high) · proprietary | 89.9% |
| 70 | Claude Sonnet 4.6 Max · proprietary | 89.9% |
| 71 | DeepSeek V4.1 Flash · 763.2B | 89.9% |
| 72 | Claude Sonnet 5 (xhigh) · proprietary | 89.5% |
| 73 | Gemini 3 Flash Preview (high) · proprietary | 89.1% |
| 74 | GPT 5.6 Luna (xhigh) · proprietary | 89.1% |
| 75 | GPT 5.6 Luna Max · proprietary | 88.8% |
| 76 | Qwen3.8 27B · 27.8B | 88.0% |
| 77 | DeepSeek V3.2 Exp · 685.4B | 87.7% |
| 78 | Grok 4.1 Fast Reasoning · proprietary | 87.7% |
| 79 | GLM 5.3 · 753.3B | 87.7% |
| 80 | Inkling · 952.4B | 87.5% |
| 81 | Qwen3.5 397B A17B · 403.4B | 87.5% |
| 82 | Qwen3.6 Max Preview · proprietary | 87.2% |
| 83 | O3 Pro (Jun 10, 2025, high) · proprietary | 86.9% |
| 84 | GPT 5.6 Luna (high) · proprietary | 86.7% |
| 85 | DeepSeek V4 Flash · 290.9B | 86.4% |
| 86 | DeepSeek V3.2 · 685.4B | 85.6% |
| 87 | O3 (Apr 16, 2025, high) · proprietary | 84.8% |
| 88 | Claude Sonnet 5 (high) · proprietary | 84.5% |
| 89 | MiMo V2.5 Pro · 1023.2B | 84.5% |
| 90 | Qwen3.5 122B A10B · 125.1B | 84.3% |
| 91 | Qwen3.7 Plus · proprietary | 84.0% |
| 92 | GPT 5.6 Sol None · proprietary | 83.7% |
| 93 | Gemini 3.5 Flash Lite (high) · proprietary | 83.5% |
| 94 | Claude Sonnet 4.5 (Sep 29, 2025, unspecified) · proprietary | 83.2% |
| 95 | GPT 5.6 Luna (medium) · proprietary | 83.2% |
| 96 | Qwen3.5 Flash · proprietary | 82.9% |
| 97 | DeepSeek V3.1 · 684.5B | 82.7% |
| 98 | Gemma 4 31B IT · 31.3B | 82.7% |
| 99 | Grok 4 Fast · proprietary | 82.7% |
| 100 | Gemini 2.5 Pro · proprietary | 82.4% |
| 101 | Qwen3.5 27B · 27.8B | 82.4% |
| 102 | Qwen3 Max (Sep 23, 2025) · proprietary | 82.1% |
| 103 | Qwen3.6 Plus · proprietary | 81.9% |
| 104 | Claude Opus 4 (May 14, 2025, unspecified) · proprietary | 81.6% |
| 105 | DeepSeek V3.1 Terminus · 684.5B | 81.3% |
| 106 | Qwen Plus (Apr 28, 2025) · proprietary | 81.1% |
| 107 | GPT 5 Mini (Aug 07, 2025, high) · proprietary | 80.5% |
| 108 | Qwen3.5 Plus · proprietary | 80.5% |
| 109 | GPT 5.4 Nano (Mar 17, 2026, xhigh) · proprietary | 80.3% |
| 110 | Qwen3 235B A22B Thinking 2507 · 235.1B | 80.3% |
| 111 | Claude Opus 4.1 (Aug 05, 2025, unspecified) · proprietary | 80.0% |
| 112 | GPT 5.4 Mini (Mar 17, 2026, xhigh) · proprietary | 80.0% |
| 113 | GPT 5.6 Luna (low) · proprietary | 80.0% |
| 114 | Qwen3.5 35B A3B · 36.0B | 80.0% |
| 115 | Claude Sonnet 5 (medium) · proprietary | 79.2% |
| 116 | MiniMax M3 · 427.0B | 78.9% |
| 117 | GPT 5.6 Terra None · proprietary | 78.7% |
| 118 | Qwen3 235B A22B Instruct 2507 · 235.1B | 78.4% |
| 119 | Qwen3.6 27B · 27.8B | 78.1% |
| 120 | O4 Mini (Apr 16, 2025, high) · proprietary | 77.6% |
| 121 | Claude Sonnet 4 (May 14, 2025, unspecified) · proprietary | 77.1% |
| 122 | Qwen3.6 Flash · proprietary | 77.1% |
| 123 | Gemini 3.1 Flash Lite (high) · proprietary | 76.8% |
| 124 | Gemini 2.5 Flash · proprietary | 76.5% |
| 125 | GPT OSS 120B · 116.8B | 76.3% |
| 126 | Qwen3 235B A22B · 235.1B | 75.7% |
| 127 | Mistral Medium 2604 · proprietary | 75.5% |
| 128 | Gemma 4 26B A4B IT · 25.8B | 74.9% |
| 129 | O1 (Dec 17, 2024, high) · proprietary | 74.7% |
| 130 | Qwen3.6 35B A3B · 36.0B | 73.9% |
| 131 | Claude Haiku 4.5 (Oct 01, 2025, unspecified) · proprietary | 73.6% |
| 132 | Claude Sonnet 5 (low) · proprietary | 73.6% |
| 133 | Qwen3.5 9B · 9.7B | 71.2% |
| 134 | Mistral Small 2603 · proprietary | 70.9% |
| 135 | Qwen3 30B A3B Thinking 2507 · 30.5B | 69.3% |
| 136 | GPT 5.6 Luna None · proprietary | 69.3% |
| 137 | GPT 4.1 Mini (Apr 14, 2025) · proprietary | 68.8% |
| 138 | O3 Mini (Jan 31, 2025, high) · proprietary | 68.8% |
| 139 | GPT 4.1 (Apr 14, 2025) · proprietary | 68.3% |
| 140 | GPT OSS 20B · 20.9B | 68.0% |
| 141 | Claude 3.5 Sonnet (Oct 22, 2024) · proprietary | 67.8% |
| 142 | Claude 3.5 Sonnet (Jun 20, 2024) · proprietary | 67.8% |
| 143 | Mistral Medium 2508 · proprietary | 67.5% |
| 144 | Qwen3 32B · 32.8B | 67.5% |
| 145 | Qwen3 30B A3B Instruct 2507 · 30.5B | 67.2% |
| 146 | Grok 2 (Dec 12) · proprietary | 65.2% |
| 147 | Mistral Large 3 675B Instruct 2512 · 675B | 65.1% |
| 148 | DeepSeek v3 0324 · 684.5B | 64.8% |
| 149 | GPT 4o (May 13, 2024) · proprietary | 64.5% |
| 150 | Qwen3 14B · 14.8B | 64.0% |
| 151 | Gemini 2.0 Flash 001 · proprietary | 63.2% |
| 152 | Qwen2.5 72B Instruct · 72.7B | 62.9% |
| 153 | Gemini 2.5 Flash Lite Preview Thinking (Jun 17) · proprietary | 62.8% |
| 154 | DeepSeek Chat · proprietary | 62.7% |
| 155 | GPT 4 (Jun 13) · proprietary | 62.7% |
| 156 | GPT 5 Nano (Aug 07, 2025, high) · proprietary | 62.7% |
| 157 | Mistral Medium 2505 · proprietary | 62.3% |
| 158 | Llama 4 Maverick 17B 128E Instruct · 401.6B | 61.9% |
| 159 | Claude 3 Opus (Feb 29, 2024) · proprietary | 61.6% |
| 160 | GPT 4 Turbo · proprietary | 61.6% |
| 161 | Llama 3.1 405B Instruct · 405.9B | 61.4% |
| 162 | C4ai Command A 03 2025 · 111.1B | 61.3% |
| 163 | Magistral Small 2509 · 24.0B | 61.3% |
| 164 | Mistral Large Instruct 2407 · 122.6B | 61.2% |
| 165 | Mistral Large Instruct 2411 · 122.6B | 60.8% |
| 166 | Qwen3 30B A3B · 30.5B | 60.3% |
| 167 | Llama 3.1 70B Instruct · 70.6B | 60.0% |
| 168 | Mistral Small 3.2 24B Instruct 2506 · 24.0B | 59.9% |
| 169 | Qwen3 8B · 8.2B | 59.7% |
| 170 | Llama 3.3 70B Instruct · 70.6B | 59.5% |
| 171 | Gemini 1.5 Pro 002 · proprietary | 59.0% |
| 172 | Mistral Small 3.1 24B Instruct 2503 · 24.0B | 58.6% |
| 173 | Llama 4 Scout 17B 16E Instruct · 108.6B | 57.9% |
| 174 | Claude 3.5 Haiku (Oct 22, 2024) · proprietary | 56.7% |
| 175 | Gemini 1.5 Pro 001 · proprietary | 56.5% |
| 176 | Mistral Large 2402 · proprietary | 55.9% |
| 177 | Mixtral 8x22B Instruct v0.1 · 140.6B | 55.1% |
| 178 | C4ai Command R Plus 08 2024 · 103.8B | 54.9% |
| 179 | GPT 4o Mini (Jul 18, 2024) · proprietary | 54.4% |
| 180 | Meta Llama 3 70B Instruct · 70.6B | 54.2% |
| 181 | Gemini 1.5 Flash 002 · proprietary | 53.8% |
| 182 | Claude 3 Sonnet (Feb 29, 2024) · proprietary | 53.6% |
| 183 | Mistral Medium 2312 · proprietary | 53.3% |
| 184 | Mistral Small 2402 · proprietary | 53.0% |
| 185 | Gemini 2.0 Flash Lite · proprietary | 52.5% |
| 186 | Gemma 3 27B IT · 27.4B | 52.5% |
| 187 | GPT 4.1 Nano (Apr 14, 2025) · proprietary | 52.5% |
| 188 | Claude 2.0 · proprietary | 51.8% |
| 189 | Ministral 3B 2410 · proprietary | 51.7% |
| 190 | Gemini 1.5 Flash 001 · proprietary | 51.4% |
| 191 | Claude 2.1 · proprietary | 50.9% |
| 192 | Gemma 3 4B IT · 4.3B | 50.9% |
| 193 | Llama 3.1 8B Instruct · 8.0B | 50.9% |
| 194 | Claude 3 Haiku (Mar 07, 2024) · proprietary | 50.1% |
| 195 | Gemini 1.5 Flash 8B 001 · proprietary | 50.0% |
| 196 | Mixtral 8x7B Instruct v0.1 · 46.7B | 49.6% |
| 197 | Mistral Small Instruct 2409 · 22.2B | 49.5% |
| 198 | Gemma 3 12B IT · 12.2B | 48.8% |
| 199 | Mistral Nemo Instruct 2407 · 12.2B | 48.6% |
| 200 | GPT 3.5 Turbo (Jan 25) · proprietary | 48.5% |
| 201 | Gemma 2 27B · proprietary | 48.0% |
| 202 | Qwen2.5 7B Instruct · 7.6B | 47.7% |
| 203 | C4ai Command R 08 2024 · 32.3B | 46.4% |
| 204 | Gemini 1.0 Pro 001 · proprietary | 45.9% |
| 205 | Claude Instant 1.2 · proprietary | 45.8% |
| 206 | Ministral 8B 2410 · proprietary | 45.7% |
| 207 | Meta Llama 3 8B Instruct · 8.0B | 43.9% |
| 208 | Open Mistral 7B · proprietary | 42.5% |
| 209 | Llama 2 13B Chat HF · 13.0B | 42.2% |
| 210 | Llama 2 70B Chat HF · 69.0B | 41.6% |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- Gemma 3 4B IT, 4B, score 50.9% — on the efficiency frontier (best score at its size or smaller).
- Qwen3 8B, 8B, score 59.7% — on the efficiency frontier (best score at its size or smaller).
- Qwen3.5 9B, 10B, score 71.2% — on the efficiency frontier (best score at its size or smaller).
- Gemma 4 26B A4B IT, 26B, score 74.9% — on the efficiency frontier (best score at its size or smaller).
- Qwen3.8 27B, 28B, score 88.0% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Flash 0731, 304B, score 90.9% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.2, 753B, score 93.6% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Pro 0813, 1.7T, score 93.9% — on the efficiency frontier (best score at its size or smaller).
DTBench: frequently asked questions
- What is the best open LLM on DTBench?
- DeepSeek V4 Pro 0813 is the top open model on DTBench, scoring 93.9%. Among all models tested — including proprietary ones — it ranks #44. The top model overall is Claude Fable 5 Max (Anthropic) at 98.4%.
- What's the best DTBench model you can run on a 24 GB GPU?
- Qwen3.8 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 15 GB), scoring 88.0% on DTBench.
- What's the best DTBench model you can run on a 12 GB GPU?
- Qwen3.5 9B is the highest-scoring open model that fits in 12 GB at 4-bit quantization (about 5 GB), scoring 71.2% on DTBench.
- Can open models match proprietary models on DTBench?
- Not quite on DTBench: the strongest proprietary model (Claude Fable 5 Max) scores 98.4%, ahead of the best open model (DeepSeek V4 Pro 0813) at 93.9% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.