Reasoning
LMCA Leaderboard
LMCA (Language Model Conceptual Argumentation) scores models on expert-rated arguments across philosophy, decision theory, and AI risk — domains where there is no empirical feedback loop to check an answer against. It is part of the Conceptual Reasoning Index, built with Anthropic by researchers Emery Cooper and Caspar Oesterheld.
Source: epoch56 open models ranked+141 proprietaryData through Oct 2026
All models ranked on LMCA
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 | Claude Sonnet 5.5 Max · proprietary | 68.3% | API only |
| 2 | Claude Opus 5.5 Max · proprietary | 68.2% | API only |
| 3 | Claude Sonnet 5.5 (xhigh) · proprietary | 66.3% | API only |
| 4 | Claude Opus 5.5 (xhigh) · proprietary | 66.2% | API only |
| 5 | Claude Sonnet 5.5 (high) · proprietary | 65.9% | API only |
| 6 | Claude Opus 5.5 (high) · proprietary | 65.9% | API only |
| 7 | Claude Opus 5.5 (medium) · proprietary | 65.7% | API only |
| 8 | Claude Fable 5.1 (unspecified) · proprietary | 65.5% | API only |
| 9 | Claude Sonnet 5.5 (medium) · proprietary | 65.5% | API only |
| 10 | Claude Fable 5.1 (medium) · proprietary | 65.3% | API only |
| 11 | Claude Fable 5.1 (high) · proprietary | 65.1% | API only |
| 12 | Claude Fable 5.1 (xhigh) · proprietary | 65.1% | API only |
| 13 | Claude Opus 5.5 (low) · proprietary | 64.9% | API only |
| 14 | Claude Sonnet 5.5 (low) · proprietary | 64.5% | API only |
| 15 | Claude Opus 5 (xhigh) · proprietary | 64.5% | API only |
| 16 | GPT 6 Astra (xhigh) · proprietary | 64.4% | API only |
| 17 | GPT 6 Astra (unspecified) · proprietary | 64.1% | API only |
| 18 | GPT 6 Astra (high) · proprietary | 63.8% | API only |
| 19 | Gemini 4 Argon (high) · proprietary | 63.7% | API only |
| 20 | Claude Opus 5 (high) · proprietary | 63.7% | API only |
| 21 | Claude Fable 5.1 (low) · proprietary | 63.5% | API only |
| 22 | Claude Opus 5 Max · proprietary | 63.3% | API only |
| 23 | Claude Opus 5 (medium) · proprietary | 62.9% | API only |
| 24 | GPT 6 Astra (medium) · proprietary | 62.9% | API only |
| 25 | Gemini 4 Argon (low) · proprietary | 62.9% | API only |
| 26 | Gemini 4 Argon (medium) · proprietary | 62.9% | API only |
| 27 | GPT 6.1 Sol Max · proprietary | 62.8% | API only |
| 28 | Claude Opus 5 (low) · proprietary | 62.6% | API only |
| 29 | GPT 6 Astra (low) · proprietary | 62.2% | API only |
| 30 | Claude Fable 5 (high) · proprietary | 61.1% | API only |
| 31 | Claude Fable 5 (xhigh) · proprietary | 61.0% | API only |
| 32 | Claude Fable 5 Max · proprietary | 60.3% | API only |
| 33 | Claude Fable 5 (medium) · proprietary | 60.3% | API only |
| 34 | Claude Fable 5 (low) · proprietary | 59.6% | API only |
| 35 | GPT 5.6 Sol Promax · proprietary | 59.2% | API only |
| 36 | GPT 6 Sol Max · proprietary | 59.1% | API only |
| 37 | GPT 5.6 Sol (xhigh) · proprietary | 58.5% | API only |
| 38 | GPT 5.6 Sol Max · proprietary | 58.4% | API only |
| 39 | Claude Opus 4.8 Max · proprietary | 57.5% | API only |
| 40 | GPT 5.6 Sol (high) · proprietary | 56.9% | API only |
| 41 | Muse Wonder 1.0 (unspecified) · proprietary | 56.8% | API only |
| 42 | GPT 5.6 Sol (medium) · proprietary | 56.4% | API only |
| 43 | GPT 5.6 Sol (low) · proprietary | 55.9% | API only |
| 44 | Claude Opus 4.6 Max · proprietary | 55.8% | API only |
| 45 | GLM 5.3 · 753.3B | 55.5% | ~456 GB |
| 46 | GPT 5.6 Terra Max · proprietary | 55.0% | API only |
| 47 | GPT 5.5 (xhigh) · proprietary | 54.3% | API only |
| 48 | GPT 5.5 Pro (xhigh) · proprietary | 53.9% | API only |
| 49 | Muse Spark 1.3 (unspecified) · proprietary | 53.9% | API only |
| 50 | Gemini 3.1 Pro Preview (high) · proprietary | 53.8% | API only |
| 51 | Gemini 3.8 Flash (medium) · proprietary | 52.9% | API only |
| 52 | Gemini 3.1 Pro Preview (medium) · proprietary | 52.8% | API only |
| 53 | GPT 5.6 Sol None · proprietary | 52.8% | API only |
| 54 | Kimi K3 · 2779.9B | 52.7% | ~1674 GB |
| 55 | GPT 5.6 Terra (xhigh) · proprietary | 52.2% | API only |
| 56 | Claude Opus 4.7 Max · proprietary | 52.2% | API only |
| 57 | GPT 5.4 (Mar 05, 2026, xhigh) · proprietary | 52.0% | API only |
| 58 | Gemini 3.8 Flash (unspecified) · proprietary | 51.5% | API only |
| 59 | GPT 5.6 Terra (high) · proprietary | 51.4% | API only |
| 60 | Gemini 3.1 Pro Preview (low) · proprietary | 51.2% | API only |
| 61 | Gemini 3.8 Flash (low) · proprietary | 51.2% | API only |
| 62 | Claude Haiku 5.5 Max · proprietary | 51.1% | API only |
| 63 | Gemini 3.7 Flash (high) · proprietary | 50.4% | API only |
| 64 | Claude Sonnet 5 (high) · proprietary | 50.0% | API only |
| 65 | Muse Spark 1.1 (high) · proprietary | 49.9% | API only |
| 66 | Claude Sonnet 5 (medium) · proprietary | 49.7% | API only |
| 67 | Claude Sonnet 5 (low) · proprietary | 49.6% | API only |
| 68 | GPT 5.6 Terra (medium) · proprietary | 49.6% | API only |
| 69 | Grok 4.7 (xhigh) · proprietary | 49.4% | API only |
| 70 | Claude Sonnet 5 Max · proprietary | 49.3% | API only |
| 71 | GPT 5.6 Terra (low) · proprietary | 49.3% | API only |
| 72 | Gemini 3.7 Flash (medium) · proprietary | 49.2% | API only |
| 73 | Claude Sonnet 5 (xhigh) · proprietary | 48.8% | API only |
| 74 | GPT 5.6 Luna (xhigh) · proprietary | 48.5% | API only |
| 75 | GPT 5.6 Luna Max · proprietary | 48.5% | API only |
| 76 | Grok 4.6 (xhigh) · proprietary | 48.5% | API only |
| 77 | Muse Spark 1.2 (xhigh) · proprietary | 48.4% | API only |
| 78 | Claude Haiku 5.5 (xhigh) · proprietary | 48.0% | API only |
| 79 | GPT 5.6 Luna (high) · proprietary | 47.4% | API only |
| 80 | Gemini 3.5 Flash (high) · proprietary | 47.1% | API only |
| 81 | Gemini 3.7 Flash (low) · proprietary | 47.1% | API only |
| 82 | DeepSeek V4.1 Flash · 763.2B | 47.0% | ~458 GB |
| 83 | Claude Sonnet 4.6 Max · proprietary | 46.5% | API only |
| 84 | GPT 5.6 Terra None · proprietary | 46.5% | API only |
| 85 | Qwen3.8 Max (xhigh) · proprietary | 46.2% | API only |
| 86 | GLM 5.2 · 753.3B | 45.8% | ~456 GB |
| 87 | Claude Haiku 5.5 (medium) · proprietary | 45.6% | API only |
| 88 | DeepSeek V4 Pro 0813 · 1650.5B | 45.5% | ~991 GB |
| 89 | Claude Haiku 5.5 (high) · proprietary | 45.2% | API only |
| 90 | Grok 4.5 (high) · proprietary | 45.2% | API only |
| 91 | Gemini 3.6 Flash (high) · proprietary | 44.9% | API only |
| 92 | Claude Haiku 5.5 (low) · proprietary | 44.8% | API only |
| 93 | GPT 6 Luna Max · proprietary | 44.5% | API only |
| 94 | Claude Opus 4.5 (Nov 01, 2025, high) · proprietary | 44.5% | API only |
| 95 | GPT 5.6 Luna (medium) · proprietary | 44.3% | API only |
| 96 | Qwen3.7 Max · proprietary | 44.0% | API only |
| 97 | GPT 5.2 (Dec 11, 2025, xhigh) · proprietary | 43.9% | API only |
| 98 | GPT 5.1 (Nov 13, 2025, high) · proprietary | 43.9% | API only |
| 99 | GPT 5.6 Luna (low) · proprietary | 43.8% | API only |
| 100 | Gemini 3 Flash Preview (high) · proprietary | 43.1% | API only |
| 101 | Qwen3.6 Max Preview · proprietary | 42.5% | API only |
| 102 | DeepSeek V4 Flash 0731 · 304.2B | 41.7% | ~183 GB |
| 103 | GPT 5.6 Luna None · proprietary | 41.4% | API only |
| 104 | Qwen3.8 27B · 27.8B | 41.4% | ~17 GB |
| 105 | DeepSeek V4 Pro · 1598.8B | 41.2% | ~960 GB |
| 106 | GPT 5.4 Mini (Mar 17, 2026, xhigh) · proprietary | 40.8% | API only |
| 107 | GPT 5 (Aug 07, 2025, high) · proprietary | 40.0% | API only |
| 108 | O3 (Apr 16, 2025, high) · proprietary | 39.7% | API only |
| 109 | Claude Haiku 5.5 None · proprietary | 39.3% | API only |
| 110 | Gemma 4 31B IT · 31.3B | 39.3% | ~20 GB |
| 111 | Claude Sonnet 4.5 (Sep 29, 2025, unspecified) · proprietary | 38.8% | API only |
| 112 | Grok 4.20 · proprietary | 38.7% | API only |
| 113 | O3 Pro (Jun 10, 2025, high) · proprietary | 38.5% | API only |
| 114 | Grok 4.3 (high) · proprietary | 38.3% | API only |
| 115 | Qwen3.5 397B A17B · 403.4B | 37.9% | ~242 GB |
| 116 | Gemini 3.5 Flash Lite (high) · proprietary | 37.6% | API only |
| 117 | Inkling · 952.4B | 37.6% | ~572 GB |
| 118 | Qwen3.7 Plus · proprietary | 37.6% | API only |
| 119 | Claude Opus 4 (May 14, 2025, unspecified) · proprietary | 37.4% | API only |
| 120 | Kimi K2.6 · 1026.9B | 37.3% | ~620 GB |
| 121 | Claude Opus 4.1 (Aug 05, 2025, unspecified) · proprietary | 37.1% | API only |
| 122 | NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B | 36.9% | ~370 GB |
| 123 | GPT 5.4 Nano (Mar 17, 2026, xhigh) · proprietary | 36.9% | API only |
| 124 | Qwen3.5 Plus · proprietary | 36.4% | API only |
| 125 | DeepSeek V4 Flash · 290.9B | 35.9% | ~175 GB |
| 126 | Mistral Large 4 1T A52B · 1000B | 35.8% | ~604 GB |
| 127 | Gemini 3.1 Flash Lite (high) · proprietary | 35.0% | API only |
| 128 | Gemini 2.5 Pro · proprietary | 34.8% | API only |
| 129 | Qwen3.6 27B · 27.8B | 34.5% | ~17 GB |
| 130 | GPT 5 Mini (Aug 07, 2025, high) · proprietary | 34.2% | API only |
| 131 | Qwen3.5 27B · 27.8B | 34.0% | ~17 GB |
| 132 | MiniMax M3 · 427.0B | 33.7% | ~257 GB |
| 133 | Qwen3.6 Plus · proprietary | 33.1% | API only |
| 134 | Qwen3.5 122B A10B · 125.1B | 32.2% | ~75 GB |
| 135 | Qwen3.6 Flash · proprietary | 31.0% | API only |
| 136 | Claude Haiku 4.5 (Oct 01, 2025, unspecified) · proprietary | 30.9% | API only |
| 137 | Qwen3.6 35B A3B · 36.0B | 29.7% | ~22 GB |
| 138 | Gemma 4 26B A4B IT · 25.8B | 29.7% | ~16 GB |
| 139 | MiMo V2.5 Pro · 1023.2B | 29.5% | ~614 GB |
| 140 | Qwen3.5 35B A3B · 36.0B | 29.5% | ~22 GB |
| 141 | Qwen3 235B A22B Thinking 2507 · 235.1B | 29.3% | ~142 GB |
| 142 | Qwen3.5 Flash · proprietary | 29.1% | API only |
| 143 | DeepSeek V3.2 Exp · 685.4B | 29.1% | ~415 GB |
| 144 | Claude Sonnet 4 (May 14, 2025, unspecified) · proprietary | 29.0% | API only |
| 145 | DeepSeek V3.2 · 685.4B | 28.8% | ~415 GB |
| 146 | DeepSeek V3.1 Terminus · 684.5B | 28.6% | ~415 GB |
| 147 | Qwen3 Max (Sep 23, 2025) · proprietary | 28.3% | API only |
| 148 | Gemini 2.5 Flash · proprietary | 27.5% | API only |
| 149 | O4 Mini (Apr 16, 2025, high) · proprietary | 26.5% | API only |
| 150 | Mistral Medium 3.5 128B · 127.7B | 26.1% | ~78 GB |
| 151 | GPT 4.1 (Apr 14, 2025) · proprietary | 25.6% | API only |
| 152 | Qwen3 235B A22B · 235.1B | 25.0% | ~142 GB |
| 153 | Qwen3.5 9B · 9.7B | 24.5% | ~6.4 GB |
| 154 | DeepSeek V3.1 · 684.5B | 24.3% | ~415 GB |
| 155 | Qwen Plus (Apr 28, 2025) · proprietary | 24.0% | API only |
| 156 | Qwen3 235B A22B Instruct 2507 · 235.1B | 23.5% | ~142 GB |
| 157 | Qwen3 30B A3B Instruct 2507 · 30.5B | 22.4% | ~19 GB |
| 158 | O1 (Dec 17, 2024, high) · proprietary | 22.3% | API only |
| 159 | GPT OSS 120B · 116.8B | 22.1% | ~71 GB |
| 160 | GPT 4.1 Mini (Apr 14, 2025) · proprietary | 21.1% | API only |
| 161 | Mistral Small 4 119B 2603 · 119.4B | 20.6% | ~73 GB |
| 162 | Qwen3 30B A3B Thinking 2507 · 30.5B | 19.5% | ~19 GB |
| 163 | Mistral Medium 2508 · proprietary | 19.1% | API only |
| 164 | O3 Mini (Jan 31, 2025, high) · proprietary | 19.0% | API only |
| 165 | Qwen3 14B · 14.8B | 18.2% | ~9.5 GB |
| 166 | Gemini 2.5 Flash Lite Preview Thinking (Jun 17) · proprietary | 18.1% | API only |
| 167 | Llama 3.3 70B Instruct · 70.6B | 17.5% | ~47 GB |
| 168 | Qwen3 32B · 32.8B | 17.3% | ~20 GB |
| 169 | GPT 4 (Jun 13) · proprietary | 17.1% | API only |
| 170 | Claude 3 Opus (Feb 29, 2024) · proprietary | 17.0% | API only |
| 171 | Mistral Large 3 675B Instruct 2512 · 675B | 16.7% | ~446 GB |
| 172 | GPT 4o (May 13, 2024) · proprietary | 16.6% | API only |
| 173 | Llama 4 Maverick 17B 128E Instruct · 401.6B | 15.9% | ~265 GB |
| 174 | Qwen3 30B A3B · 30.5B | 15.8% | ~19 GB |
| 175 | DeepSeek v3 0324 · 684.5B | 15.5% | ~415 GB |
| 176 | DeepSeek Chat · proprietary | 15.2% | API only |
| 177 | Llama 3.1 70B Instruct · 70.6B | 14.8% | ~47 GB |
| 178 | GPT OSS 20B · 20.9B | 14.5% | ~13 GB |
| 179 | Qwen2.5 72B Instruct · 72.7B | 13.4% | ~45 GB |
| 180 | Gemma 3 27B IT · 27.4B | 12.3% | ~18 GB |
| 181 | Llama 4 Scout 17B 16E Instruct · 108.6B | 12.0% | ~72 GB |
| 182 | GPT 4o Mini (Jul 18, 2024) · proprietary | 10.4% | API only |
| 183 | C4ai Command A 03 2025 · 111.1B | 10.3% | ~73 GB |
| 184 | GPT 4 Turbo · proprietary | 9.8% | API only |
| 185 | GPT 3.5 Turbo (Jan 25) · proprietary | 9.7% | API only |
| 186 | C4ai Command R 08 2024 · 32.3B | 9.2% | ~21 GB |
| 187 | Claude 3 Haiku (Mar 07, 2024) · proprietary | 8.8% | API only |
| 188 | Qwen3 8B · 8.2B | 8.8% | ~5.5 GB |
| 189 | GPT 5 Nano (Aug 07, 2025, high) · proprietary | 7.9% | API only |
| 190 | Gemma 2 27B · proprietary | 7.1% | API only |
| 191 | Qwen2.5 7B Instruct · 7.6B | 6.4% | ~5.0 GB |
| 192 | Ministral 3B 2410 · proprietary | 5.5% | API only |
| 193 | GPT 4.1 Nano (Apr 14, 2025) · proprietary | 5.5% | API only |
| 194 | Llama 3.1 8B Instruct · 8.0B | 5.4% | ~5.3 GB |
| 195 | C4ai Command R Plus 08 2024 · 103.8B | 5.0% | ~69 GB |
| 196 | Gemma 3 12B IT · 12.2B | 4.5% | ~8.0 GB |
| 197 | Gemma 3 4B IT · 4.3B | 2.8% | ~2.8 GB |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- Gemma 3 4B IT, 4B, score 2.8% — on the efficiency frontier (best score at its size or smaller).
- Qwen2.5 7B Instruct, 8B, score 6.4% — on the efficiency frontier (best score at its size or smaller).
- Qwen3 8B, 8B, score 8.8% — on the efficiency frontier (best score at its size or smaller).
- Qwen3.5 9B, 10B, score 24.5% — on the efficiency frontier (best score at its size or smaller).
- Gemma 4 26B A4B IT, 26B, score 29.7% — on the efficiency frontier (best score at its size or smaller).
- Qwen3.8 27B, 28B, score 41.4% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Flash 0731, 304B, score 41.7% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.3, 753B, score 55.5% — on the efficiency frontier (best score at its size or smaller).
LMCA: frequently asked questions
- What is the best open LLM on LMCA?
- GLM 5.3 is the top open model on LMCA, scoring 55.5%. Among all models tested — including proprietary ones — it ranks #45. The top model overall is Claude Sonnet 5.5 Max (Anthropic) at 68.3%.
- What's the best LMCA model you can run on a 24 GB GPU?
- Qwen3.8 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (~17 GB), scoring 41.4% on LMCA.
- What's the best LMCA model you can run on a 12 GB GPU?
- Qwen3.5 9B is the highest-scoring open model that fits in 12 GB at 4-bit quantization (~6.4 GB), scoring 24.5% on LMCA.
- Can open models match proprietary models on LMCA?
- Not quite on LMCA: the strongest proprietary model (Claude Sonnet 5.5 Max) scores 68.3%, ahead of the best open model (GLM 5.3) at 55.5% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.