Reasoning
Chess Puzzles Leaderboard
Chess Puzzles is an Epoch AI evaluation that asks models to find the winning move in tactical chess positions drawn from real games. Because each puzzle has one verifiable correct answer and no natural-language shortcut, it isolates concrete multi-step planning.
Source: epoch16 open models ranked+138 proprietaryData through Nov 2026
Open models ranked on Chess Puzzles
# shows rank among open models / rank overall (including proprietary).
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 / 20 | DeepSeek V4.1 Flash · 763.2B | 48.0% | ~458 GB |
| 2 / 26 | DeepSeek V4 Pro 0813 · 1650.5B | 46.0% | ~991 GB |
| 3 / 27 | DeepSeek V4.1 Flash 1017 · 763.2B | 46.0% | ~458 GB |
| 4 / 46 | GLM 5.3 · 753.3B | 37.0% | ~456 GB |
| 5 / 48 | Kimi K3 · 2779.9B | 37.0% | ~1674 GB |
| 6 / 65 | Mistral Large 4 1T A52B · 1000B | 33.0% | ~604 GB |
| 7 / 66 | DeepSeek V4 Flash 0731 · 304.2B | 32.0% | ~183 GB |
| 8 / 88 | GLM 5.2 · 753.3B | 22.0% | ~456 GB |
| 9 / 94 | GLM 4.7 · 358.3B | 21.0% | ~216 GB |
| 10 / 95 | GLM 5.3 Flash · 321.3B | 21.0% | ~195 GB |
| 11 / 96 | Inkling · 952.4B | 21.0% | ~572 GB |
| 12 / 99 | GLM 5.1 · 753.9B | 20.0% | ~457 GB |
| 13 / 101 | Kimi K2.6 · 1026.9B | 20.0% | ~620 GB |
| 14 / 104 | DeepSeek V4 Pro · 1598.8B | 19.0% | ~960 GB |
| 15 / 109 | MiniMax M3 · 427.0B | 17.0% | ~257 GB |
| 16 / 118 | Kimi K2.7 Code · 1026.9B | 14.0% | ~620 GB |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- DeepSeek V4 Flash 0731, 304B, score 32.0% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.3, 753B, score 37.0% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4.1 Flash, 763B, score 48.0% — on the efficiency frontier (best score at its size or smaller).
Chess Puzzles: frequently asked questions
- What is the best open LLM on Chess Puzzles?
- DeepSeek V4.1 Flash is the top open model on Chess Puzzles, scoring 48.0%. Among all models tested — including proprietary ones — it ranks #19. The top model overall is GPT 6 Astra Promax (OpenAI) at 76.0%.
- Can open models match proprietary models on Chess Puzzles?
- Not quite on Chess Puzzles: the strongest proprietary model (GPT 6 Astra Promax) scores 76.0%, ahead of the best open model (DeepSeek V4.1 Flash) at 48.0% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.