llmrun
Reasoning

SimpleBench Leaderboard

SimpleBench is a set of everyday, common-sense and trick questions that humans answer easily but language models often get wrong. It probes basic reasoning and robustness rather than specialist knowledge.

Source: epoch23 open models ranked+78 proprietaryData through Aug 2026

Open modelsAll models

All models ranked on SimpleBench

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScore
1Claude Fable 5 Max · proprietary
81.9%
2Claude Opus 5 (unspecified) · proprietary
80.6%
3Gemini 3.1 Pro Preview · proprietary
79.6%
4GPT 5.5 Pro (high) · proprietary
76.9%
5GPT 5.5 Pro (unspecified) · proprietary
76.9%
6Gemini 3.5 Flash (unspecified) · proprietary
76.7%
7Gemini 3 Pro Preview · proprietary
76.4%
8Grok 4.6 (unspecified) · proprietary
75.9%
9Muse Spark 1.2 · proprietary
74.5%
10GPT 5.4 Pro (Mar 05, 2026, unspecified) · proprietary
74.1%
11GPT 5.6 Sol Prounknown · proprietary
71.7%
12GPT 5.6 Sol Proxhigh · proprietary
71.7%
13Qwen3.7 Max · proprietary
70.4%
14Grok 4.5 (unspecified) · proprietary
70.0%
15GPT 5.5 (unspecified) · proprietary
69.0%
16Claude Opus 4.6 · proprietary
67.6%
17Claude Opus 4.8 (unspecified) · proprietary
64.8%
18GPT 5.6 Sol (unspecified) · proprietary
64.8%
19GPT 5.6 Sol (xhigh) · proprietary
64.8%
20Qwen3.6 Max Preview · proprietary
63.0%
21Gemini 2.5 Pro Preview (Jun 05) · proprietary
62.4%
22Claude Opus 4.5 (Nov 01, 2025, unspecified) · proprietary
62.0%
23Claude Opus 4.5 (Nov 01, 2025) · proprietary
62.0%
24Claude Opus 4.7 (unspecified) · proprietary
61.7%
25GPT 5 Pro (Oct 06, 2025, high) · proprietary
61.6%
26GPT 5 Pro (Oct 06, 2025, unspecified) · proprietary
61.6%
27DeepSeek V4 Flash 0731 · 304.2B
61.1%
28Gemini 3 Flash Preview · proprietary
61.1%
29Kimi K3 · 2779.9B
60.7%
30Claude Sonnet 5 (unspecified) · proprietary
60.6%
31Grok 4 (Jul 09) · proprietary
60.5%
32Claude Opus 4.1 (Aug 05, 2025, unspecified) · proprietary
60.0%
33Claude Opus 4.1 (Aug 05, 2025) · proprietary
60.0%
34Claude Opus 4 (May 14, 2025, 12K) · proprietary
58.8%
35Claude Opus 4 (May 14, 2025, unspecified) · proprietary
58.8%
36GLM 5.2 · 753.3B
58.8%
37Kimi K2.7 Code · 1026.9B
57.9%
38GPT 5.2 Pro (Dec 11, 2025) · proprietary
57.4%
39GPT 5 (Aug 07, 2025, high) · proprietary
56.7%
40Grok 4.1 Fast Non Reasoning · proprietary
56.0%
41Grok 4.1 Fast Reasoning · proprietary
56.0%
42GLM 5.1 · 753.9B
55.1%
43Claude Sonnet 4.5 (Sep 29, 2025, 12K) · proprietary
54.3%
44Claude Sonnet 4.5 (Sep 29, 2025, unspecified) · proprietary
54.3%
45GLM 5 · 753.9B
53.2%
46GPT 5.1 (Nov 13, 2025, high) · proprietary
53.2%
47O3 (Apr 16, 2025, high) · proprietary
53.1%
48DeepSeek V3.2 Speciale · 685.4B
52.6%
49Gemini 2.5 Pro Exp (Mar 25) · proprietary
51.6%
50Gemini 2.5 Pro Preview (Mar 25) · proprietary
51.6%
51Inkling · 952.4B
50.0%
52GPT 5.6 Terra (unspecified) · proprietary
48.9%
53GPT 5.6 Terra (xhigh) · proprietary
48.9%
54GLM 4.7 · 358.3B
47.7%
55GPT 5.6 Luna (unspecified) · proprietary
46.8%
56GPT 5.6 Luna (xhigh) · proprietary
46.8%
57Kimi K2.5 · 1026.9B
46.8%
58Claude 3.7 Sonnet (Feb 19, 2025, 12K) · proprietary
46.4%
59Claude 3.7 Sonnet (Feb 19, 2025, unspecified) · proprietary
46.4%
60DeepSeek V4 Flash · 290.9B
46.3%
61GPT 5.2 (Dec 11, 2025, high) · proprietary
45.8%
62GPT 5.2 (Dec 11, 2025, unspecified) · proprietary
45.8%
63MiniMax M3 · 427.0B
45.8%
64Claude Sonnet 4 (May 14, 2025, 12K) · proprietary
45.5%
65Claude Sonnet 4 (May 14, 2025, unspecified) · proprietary
45.5%
66Claude 3.7 Sonnet (Feb 19, 2025) · proprietary
44.9%
67O1 Preview (Sep 12, 2024) · proprietary
41.7%
68Claude 3.5 Sonnet (Oct 22, 2024) · proprietary
41.4%
69Gemini 2.5 Flash · proprietary
41.2%
70DeepSeek R1 0528 · 684.5B
40.8%
71O1 (Dec 17, 2024, high) · proprietary
40.1%
72DeepSeek V3.1 · 684.5B
40.0%
73O4 Mini (Apr 16, 2025, high) · proprietary
38.7%
74O1 (Dec 17, 2024, medium) · proprietary
36.7%
75Grok 3 · proprietary
36.1%
76Qwen3.6 Flash · proprietary
35.2%
77GPT 4.5 Preview (Feb 27, 2025) · proprietary
34.5%
78Gemini Exp (Dec 06) · proprietary
31.1%
79Qwen3 235B A22B · 235.1B
31.0%
80DeepSeek R1 · 684.5B
30.9%
81Gemini 2.0 Flash Thinking Exp (Jan 21) · proprietary
30.7%
82Llama 4 Maverick 17B 128E Instruct · 401.6B
27.7%
83Claude 3.5 Sonnet (Jun 20, 2024) · proprietary
27.5%
84DeepSeek v3 0324 · 684.5B
27.2%
85Gemini 1.5 Pro 002 · proprietary
27.1%
86GPT 4.1 (Apr 14, 2025) · proprietary
27.0%
87Kimi K2 Instruct · 1026.4B
26.3%
88GPT 4 Turbo (Apr 09, 2024) · proprietary
25.1%
89Claude 3 Opus (Feb 29, 2024) · proprietary
23.5%
90Llama 3.1 405B Instruct · 405.9B
23.0%
91O3 Mini (Jan 31, 2025, high) · proprietary
22.8%
92Grok 2 (Dec 12) · proprietary
22.7%
93Mistral Large 2407 · proprietary
22.5%
94GPT OSS 120B · 116.8B
22.1%
95Llama 3.3 70B Instruct · 70.6B
19.9%
96DeepSeek v3 · 684.5B
18.9%
97Gemini 2.0 Flash Exp · proprietary
18.9%
98O1 Mini (Sep 12, 2024, medium) · proprietary
18.1%
99GPT 4o (Aug 06, 2024) · proprietary
17.8%
100C4ai Command R Plus (Aug 2024) · proprietary
17.4%
101GPT 4o Mini (Jul 18, 2024) · proprietary
10.7%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →61.1%18.9%Kimi K3 · 2.8T · 60.7%GLM 5.2 · 753B · 58.8%Kimi K2.7 Code · 1T · 57.9%GLM 5.1 · 754B · 55.1%GLM 5 · 754B · 53.2%DeepSeek V3.2 Speciale · 685B · 52.6%Inkling · 952B · 50.0%GLM 4.7 · 358B · 47.7%Kimi K2.5 · 1T · 46.8%MiniMax M3 · 427B · 45.8%DeepSeek R1 0528 · 685B · 40.8%DeepSeek V3.1 · 685B · 40.0%DeepSeek R1 · 685B · 30.9%Llama 4 Maverick 17B 128E Instruct · 402B · 27.7%DeepSeek v3 0324 · 685B · 27.2%Kimi K2 Instruct · 1T · 26.3%Llama 3.1 405B Instruct · 406B · 23.0%DeepSeek v3 · 685B · 18.9%Llama 3.3 70B Instruct · 71B · 19.9%Llama 3.3 70B InstructGPT OSS 120B · 117B · 22.1%GPT OSS 120BQwen3 235B A22B · 235B · 31.0%Qwen3 235B A22BDeepSeek V4 Flash · 291B · 46.3%DeepSeek V4 FlashDeepSeek V4 Flash 0731 · 304B · 61.1%DeepSeek V4 Flash 0731
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Llama 3.3 70B Instruct, 71B, score 19.9% — on the efficiency frontier (best score at its size or smaller).
  • GPT OSS 120B, 117B, score 22.1% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3 235B A22B, 235B, score 31.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Flash, 291B, score 46.3% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Flash 0731, 304B, score 61.1% — on the efficiency frontier (best score at its size or smaller).

SimpleBench: frequently asked questions

What is the best open LLM on SimpleBench?
DeepSeek V4 Flash 0731 is the top open model on SimpleBench, scoring 61.1%. Among all models tested — including proprietary ones — it ranks #27. The top model overall is Claude Fable 5 Max (Anthropic) at 81.9%.
Can open models match proprietary models on SimpleBench?
Not quite on SimpleBench: the strongest proprietary model (Claude Fable 5 Max) scores 81.9%, ahead of the best open model (DeepSeek V4 Flash 0731) at 61.1% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.