Coding
MirrorCode Leaderboard
MirrorCode, from Epoch AI and METR, asks coding agents to reimplement entire programs without seeing their source code. Agents can probe the original program, and their implementations must reproduce its outputs on end-to-end tests, including held-out cases. This leaderboard uses Epoch's best score across scorers.
Source: epoch5 open models ranked+41 proprietaryData through Oct 2026
All models ranked on MirrorCode
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 | Claude Sonnet 5.5 Max · proprietary | 97.8% | API only |
| 2 | Claude Opus 5.5 Max · proprietary | 95.6% | API only |
| 3 | Claude Fable 5.1 Max · proprietary | 93.3% | API only |
| 4 | Claude Opus 5 Max · proprietary | 87.8% | API only |
| 5 | Claude Haiku 5.5 Max · proprietary | 78.9% | API only |
| 6 | Claude Opus 5.5 (high) · proprietary | 78.9% | API only |
| 7 | GPT 6.1 Terra Max · proprietary | 76.7% | API only |
| 8 | Claude Fable 5.1 (high) · proprietary | 73.3% | API only |
| 9 | Claude Fable 5 Max · proprietary | 70.0% | API only |
| 10 | Claude Opus 4.8 Max · proprietary | 65.6% | API only |
| 11 | DeepSeek V4.1 Flash · 763.2B | 64.4% | ~458 GB |
| 12 | Claude Fable 5 (high) · proprietary | 63.9% | API only |
| 13 | Claude Sonnet 5 Max · proprietary | 63.3% | API only |
| 14 | GPT 6.1 Sol Max · proprietary | 61.1% | API only |
| 15 | GPT 6 Astra Max · proprietary | 60.0% | API only |
| 16 | Gemini 4 Argon (high) · proprietary | 58.9% | API only |
| 17 | Grok 4.7 (xhigh) · proprietary | 54.4% | API only |
| 18 | Kimi K3 · 2779.9B | 54.4% | ~1674 GB |
| 19 | Muse Spark 1.3 Max · proprietary | 54.4% | API only |
| 20 | Qwen3.8 Max (xhigh) · proprietary | 52.2% | API only |
| 21 | GPT 6 Sol Max · proprietary | 48.9% | API only |
| 22 | Qwen3.8 Max (Sep 02, xhigh) · proprietary | 47.8% | API only |
| 23 | GPT 6 Astra (high) · proprietary | 46.7% | API only |
| 24 | Muse Wonder 1.0 Max · proprietary | 45.6% | API only |
| 25 | Claude Opus 4.7 Max · proprietary | 43.3% | API only |
| 26 | Gemini 3.7 Flash (high) · proprietary | 41.1% | API only |
| 27 | GPT 5.6 Terra Max · proprietary | 41.1% | API only |
| 28 | Grok 4.6 (xhigh) · proprietary | 41.1% | API only |
| 29 | Mistral Large 4 1T A52B · 1000B | 40.0% | ~604 GB |
| 30 | Gemini 3.8 Flash (high) · proprietary | 38.9% | API only |
| 31 | GLM 5.2 · 753.3B | 36.7% | ~456 GB |
| 32 | Claude Opus 4.6 Max · proprietary | 33.3% | API only |
| 33 | DeepSeek V4 Pro 0813 · 1650.5B | 33.3% | ~991 GB |
| 34 | GPT 5.2 (Dec 11, 2025, xhigh) · proprietary | 32.2% | API only |
| 35 | Grok 4.5 (high) · proprietary | 32.2% | API only |
| 36 | Claude Opus 4.7 (high) · proprietary | 31.1% | API only |
| 37 | GPT 6 Luna Max · proprietary | 26.7% | API only |
| 38 | Gemini 3.6 Flash (high) · proprietary | 23.3% | API only |
| 39 | GPT 5.4 (Mar 05, 2026, xhigh) · proprietary | 23.3% | API only |
| 40 | GPT 5.5 (xhigh) · proprietary | 23.3% | API only |
| 41 | GPT 5.6 Luna Max · proprietary | 21.1% | API only |
| 42 | GPT 5.6 Sol (high) · proprietary | 20.0% | API only |
| 43 | GPT 5.4 (Mar 05, 2026, high) · proprietary | 15.6% | API only |
| 44 | GPT 5.5 (high) · proprietary | 10.0% | API only |
| 45 | Gemini 3.1 Pro Preview (high) · proprietary | 8.9% | API only |
| 46 | GPT 5 (Aug 07, 2025, high) · proprietary | 4.4% | API only |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- GLM 5.2, 753B, score 36.7% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4.1 Flash, 763B, score 64.4% — on the efficiency frontier (best score at its size or smaller).
MirrorCode: frequently asked questions
- What is the best open LLM on MirrorCode?
- DeepSeek V4.1 Flash is the top open model on MirrorCode, scoring 64.4%. Among all models tested — including proprietary ones — it ranks #11. The top model overall is Claude Sonnet 5.5 Max (Anthropic) at 97.8%.
- Can open models match proprietary models on MirrorCode?
- Not quite on MirrorCode: the strongest proprietary model (Claude Sonnet 5.5 Max) scores 97.8%, ahead of the best open model (DeepSeek V4.1 Flash) at 64.4% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.