llmrun
Reasoning

Chess Puzzles Leaderboard

Chess Puzzles is an Epoch AI evaluation that asks models to find the winning move in tactical chess positions drawn from real games. Because each puzzle has one verifiable correct answer and no natural-language shortcut, it isolates concrete multi-step planning.

Source: epoch16 open models ranked+138 proprietaryData through Nov 2026

Open modelsAll models

All models ranked on Chess Puzzles

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScoreVRAM @ Q4
1GPT 6 Astra Promax · proprietary
76.0%
API only
2GPT 6 Astra Max · proprietary
74.0%
API only
3GPT 5.6 Sol Promax · proprietary
68.0%
API only
4GPT 6.1 Sol Max · proprietary
68.0%
API only
5Claude Opus 5.5 Max · proprietary
67.0%
API only
6Gemini 3.7 Flash (high) · proprietary
66.0%
API only
7GPT 5.5 (xhigh) · proprietary
63.0%
API only
8GPT 5.5 Pro (xhigh) · proprietary
62.0%
API only
9GPT 5.2 Pro (Dec 11, 2025, xhigh) · proprietary
60.0%
API only
10Claude Sonnet 5.5 Max · proprietary
59.0%
API only
11Gemini 4 Argon (high) · proprietary
59.0%
API only
12GPT 5.4 Pro (Mar 05, 2026, xhigh) · proprietary
59.0%
API only
13GPT 5.6 Sol Max · proprietary
59.0%
API only
14Gemini 3.8 Flash (high) · proprietary
58.0%
API only
15GPT 6 Sol Max · proprietary
56.0%
API only
16Claude Haiku 5.5 Max · proprietary
55.0%
API only
17Claude Opus 4.8 Max · proprietary
52.0%
API only
18Gemini 3.1 Pro Preview (high) · proprietary
51.0%
API only
19Claude Fable 5.1 Max · proprietary
48.0%
API only
20DeepSeek V4.1 Flash · 763.2B
48.0%
~458 GB
21GPT 5 Pro (Oct 06, 2025, high) · proprietary
48.0%
API only
22GPT 5.2 (Dec 11, 2025, xhigh) · proprietary
48.0%
API only
23GPT 5.4 (Mar 05, 2026, xhigh) · proprietary
48.0%
API only
24Claude Fable 5 Max · proprietary
47.0%
API only
25GPT 5.6 Terra Max · proprietary
47.0%
API only
26DeepSeek V4 Pro 0813 · 1650.5B
46.0%
~991 GB
27DeepSeek V4.1 Flash 1017 · 763.2B
46.0%
~458 GB
28Qwen3.8 Max (xhigh) · proprietary
46.0%
API only
29Claude Fable 5 (high) · proprietary
45.0%
API only
30Claude Sonnet 5 Max · proprietary
45.0%
API only
31Gemini 3.5 Flash (high) · proprietary
45.0%
API only
32GPT 5.2 (Dec 11, 2025, high) · proprietary
45.0%
API only
33GPT 5.4 Nano (Mar 17, 2026, xhigh) · proprietary
45.0%
API only
34Kimi K3.1 Pro Together Max · proprietary
45.0%
API only
35Claude Opus 5 Max · proprietary
43.0%
API only
36GPT 5.6 Luna Max · proprietary
43.0%
API only
37Gemini 3.5 Flash (low) · proprietary
42.0%
API only
38Gemini 3.6 Flash (low) · proprietary
42.0%
API only
39GPT 5.1 (Nov 13, 2025, high) · proprietary
42.0%
API only
40GPT 5.4 (Mar 05, 2026, high) · proprietary
42.0%
API only
41Claude Opus 4.7 Max · proprietary
41.0%
API only
42Grok 4.7 (xhigh) · proprietary
41.0%
API only
43Doubao Seed Evolving (high) · proprietary
40.0%
API only
44Gemini 3 Flash Preview (high) · proprietary
40.0%
API only
45Gemini 3.6 Flash (high) · proprietary
40.0%
API only
46GLM 5.3 · 753.3B
37.0%
~456 GB
47GPT 5.4 (Mar 05, 2026, medium) · proprietary
37.0%
API only
48Kimi K3 · 2779.9B
37.0%
~1674 GB
49Muse Spark 1.3 Max · proprietary
37.0%
API only
50O3 (Apr 16, 2025, medium) · proprietary
37.0%
API only
51Qwen3.8 Max (Sep 02, xhigh) · proprietary
37.0%
API only
52Claude Opus 4.7 (xhigh) · proprietary
36.0%
API only
53GPT 5 (Aug 07, 2025, high) · proprietary
36.0%
API only
54GPT 5.2 (Dec 11, 2025, medium) · proprietary
36.0%
API only
55GPT 5.4 Nano (Mar 17, 2026, high) · proprietary
36.0%
API only
56GPT 6 Luna Max · proprietary
36.0%
API only
57O3 (Apr 16, 2025, high) · proprietary
36.0%
API only
58O3 (Apr 16, 2025, low) · proprietary
36.0%
API only
59Grok 4.5 (high) · proprietary
35.0%
API only
60Claude Sonnet 5 (xhigh) · proprietary
34.0%
API only
61Muse Spark 1.1 (xhigh) · proprietary
34.0%
API only
62Muse Spark 1.3 (xhigh) · proprietary
34.0%
API only
63Grok 4.6 (high) · proprietary
33.0%
API only
64Grok 4.6 (xhigh) · proprietary
33.0%
API only
65Mistral Large 4 1T A52B · 1000B
33.0%
~604 GB
66DeepSeek V4 Flash 0731 · 304.2B
32.0%
~183 GB
67GPT 5 (Aug 07, 2025, medium) · proprietary
32.0%
API only
68Claude Fable 5 (low) · proprietary
31.0%
API only
69GPT 5.6 Sol (low) · proprietary
31.0%
API only
70GPT 5.5 (low) · proprietary
30.0%
API only
71Claude Opus 4.8 (low) · proprietary
29.0%
API only
72Gemini 3.6 Flash (minimal) · proprietary
29.0%
API only
73Grok 4.20 0309 Reasoning · proprietary
29.0%
API only
74Claude Opus 5 (low) · proprietary
27.0%
API only
75Gemini 3.5 Flash (minimal) · proprietary
27.0%
API only
76GPT 5 (Aug 07, 2025, low) · proprietary
27.0%
API only
77GPT 5.6 Terra (low) · proprietary
27.0%
API only
78Muse Spark 1.2 (xhigh) · proprietary
27.0%
API only
79Qwen3.7 Max · proprietary
27.0%
API only
80Claude Opus 4.7 (low) · proprietary
26.0%
API only
81Doubao Seed 2.1 Turbo 260628 (high) · proprietary
26.0%
API only
82GPT 5.2 (Dec 11, 2025, low) · proprietary
26.0%
API only
83Claude Haiku 5.5 (low) · proprietary
24.0%
API only
84GPT 5 Mini (Aug 07, 2025, high) · proprietary
24.0%
API only
85Claude Sonnet 4.6 Max · proprietary
23.0%
API only
86GPT 5.1 (Nov 13, 2025, low) · proprietary
23.0%
API only
87GPT 5.4 Mini (Mar 17, 2026, high) · proprietary
23.0%
API only
88GLM 5.2 · 753.3B
22.0%
~456 GB
89GPT 5 Nano (Aug 07, 2025, high) · proprietary
22.0%
API only
90O4 Mini (Apr 16, 2025, medium) · proprietary
22.0%
API only
91Claude Opus 4.6 Max · proprietary
21.0%
API only
92Doubao Seed 2.1 Pro 260628 (high) · proprietary
21.0%
API only
93Gemini 3.5 Flash Lite (low) · proprietary
21.0%
API only
94GLM 4.7 · 358.3B
21.0%
~216 GB
95GLM 5.3 Flash · 321.3B
21.0%
~195 GB
96Inkling · 952.4B
21.0%
~572 GB
97Doubao Seed 2.0 Lite 260428 (high) · proprietary
20.0%
API only
98Doubao Seed 2.0 Pro 260215 (high) · proprietary
20.0%
API only
99GLM 5.1 · 753.9B
20.0%
~457 GB
100GPT 5.4 Nano (Mar 17, 2026, low) · proprietary
20.0%
API only
101Kimi K2.6 · 1026.9B
20.0%
~620 GB
102Claude Opus 4.8 None · proprietary
19.0%
API only
103Claude Sonnet 4.6 (high) · proprietary
19.0%
API only
104DeepSeek V4 Pro · 1598.8B
19.0%
~960 GB
105GPT 5.4 (Mar 05, 2026, low) · proprietary
19.0%
API only
106Grok 4.3 (high) · proprietary
18.0%
API only
107GPT 5.4 Mini (Mar 17, 2026, xhigh) · proprietary
17.0%
API only
108GPT 5.6 Luna (low) · proprietary
17.0%
API only
109MiniMax M3 · 427.0B
17.0%
~257 GB
110Claude Sonnet 4.6 (medium) · proprietary
16.0%
API only
111Gemini 3.5 Flash Lite (high) · proprietary
16.0%
API only
112GPT 5 Mini (Aug 07, 2025, low) · proprietary
16.0%
API only
113Qwen3.6 Plus · proprietary
16.0%
API only
114GPT 5 Nano (Aug 07, 2025, low) · proprietary
15.0%
API only
115O3 Mini (Jan 31, 2025, high) · proprietary
15.0%
API only
116O4 Mini (Apr 16, 2025, low) · proprietary
15.0%
API only
117Claude Opus 4.5 (Nov 01, 2025, 59K) · proprietary
14.0%
API only
118Kimi K2.7 Code · 1026.9B
14.0%
~620 GB
119Qwen3.5 Flash · proprietary
14.0%
API only
120O4 Mini (Apr 16, 2025, high) · proprietary
13.0%
API only
121Qwen3.6 Flash · proprietary
12.0%
API only
122GPT 4.1 (Apr 14, 2025) · proprietary
11.0%
API only
123GPT 5 (Aug 07, 2025, minimal) · proprietary
11.0%
API only
124O1 (Dec 17, 2024, low) · proprietary
11.0%
API only
125O3 Mini (Jan 31, 2025, medium) · proprietary
11.0%
API only
126Claude Sonnet 4.5 (Sep 29, 2025, 32K) · proprietary
10.0%
API only
127Claude Sonnet 4.5 (Sep 29, 2025, 59K) · proprietary
10.0%
API only
128O1 (Dec 17, 2024, high) · proprietary
10.0%
API only
129GPT 4o (Aug 06, 2024) · proprietary
9.0%
API only
130GPT 5.5 None · proprietary
9.0%
API only
131Qwen3.5 Plus · proprietary
9.0%
API only
132Claude Haiku 4.5 (Oct 01, 2025, 59K) · proprietary
8.0%
API only
133O1 (Dec 17, 2024, medium) · proprietary
8.0%
API only
134Gemini 3.5 Flash Lite (minimal) · proprietary
7.0%
API only
135GPT 5.6 Sol None · proprietary
7.0%
API only
136Claude Sonnet 4.5 (Sep 29, 2025) · proprietary
6.0%
API only
137GPT 5.1 2025 11.13 None · proprietary
6.0%
API only
138O3 Mini (Jan 31, 2025, low) · proprietary
6.0%
API only
139Claude Haiku 4.5 (Oct 01, 2025, 32K) · proprietary
5.0%
API only
140Claude Opus 4.5 (Nov 01, 2025, 32K) · proprietary
5.0%
API only
141Claude Opus 4.5 (Nov 01, 2025) · proprietary
5.0%
API only
142GPT 5.6 Terra None · proprietary
5.0%
API only
143Seed 1.6 · proprietary
5.0%
API only
144Claude 3 Opus (Feb 29, 2024) · proprietary
4.0%
API only
145GPT 4o Mini (Jul 18, 2024) · proprietary
3.0%
API only
146GPT 5.4 2026 03.05 None · proprietary
3.0%
API only
147GPT 5.6 Luna None · proprietary
3.0%
API only
148GPT 5 Nano (Aug 07, 2025, minimal) · proprietary
2.0%
API only
149GPT 5.2 2025 12.11 None · proprietary
2.0%
API only
150GPT 5.4 Mini 2026 03.17 None · proprietary
2.0%
API only
151GPT 5.4 Nano 2026 03.17 None · proprietary
2.0%
API only
152GPT 4.1 Mini (Apr 14, 2025) · proprietary
1.0%
API only
153GPT 4.1 Nano (Apr 14, 2025) · proprietary
0.0%
API only
154GPT 5 Mini (Aug 07, 2025, minimal) · proprietary
0.0%
API only

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

304B2.8Tmodel size (log scale) →48.0%14.0%DeepSeek V4.1 Flash 1017 · 763B · 46.0%DeepSeek V4 Pro 0813 · 1.7T · 46.0%Kimi K3 · 2.8T · 37.0%Mistral Large 4 1T A52B · 1T · 33.0%GLM 5.2 · 753B · 22.0%Inkling · 952B · 21.0%GLM 4.7 · 358B · 21.0%GLM 5.3 Flash · 321B · 21.0%Kimi K2.6 · 1T · 20.0%GLM 5.1 · 754B · 20.0%DeepSeek V4 Pro · 1.6T · 19.0%MiniMax M3 · 427B · 17.0%Kimi K2.7 Code · 1T · 14.0%DeepSeek V4 Flash 0731 · 304B · 32.0%DeepSeek V4 Flash 0731GLM 5.3 · 753B · 37.0%GLM 5.3DeepSeek V4.1 Flash · 763B · 48.0%DeepSeek V4.1 Flash
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • DeepSeek V4 Flash 0731, 304B, score 32.0% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3, 753B, score 37.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4.1 Flash, 763B, score 48.0% — on the efficiency frontier (best score at its size or smaller).

Chess Puzzles: frequently asked questions

What is the best open LLM on Chess Puzzles?
DeepSeek V4.1 Flash is the top open model on Chess Puzzles, scoring 48.0%. Among all models tested — including proprietary ones — it ranks #19. The top model overall is GPT 6 Astra Promax (OpenAI) at 76.0%.
Can open models match proprietary models on Chess Puzzles?
Not quite on Chess Puzzles: the strongest proprietary model (GPT 6 Astra Promax) scores 76.0%, ahead of the best open model (DeepSeek V4.1 Flash) at 48.0% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.