Coding
MirrorCode Leaderboard
MirrorCode, from Epoch AI and METR, asks coding agents to reimplement entire programs without seeing their source code. Agents can probe the original program, and their implementations must reproduce its outputs on end-to-end tests, including held-out cases. This leaderboard uses Epoch's best score across scorers.
Source: epoch5 open models ranked+41 proprietaryData through Oct 2026
Open models ranked on MirrorCode
# shows rank among open models / rank overall (including proprietary).
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 / 11 | DeepSeek V4.1 Flash · 763.2B | 64.4% | ~458 GB |
| 2 / 18 | Kimi K3 · 2779.9B | 54.4% | ~1674 GB |
| 3 / 29 | Mistral Large 4 1T A52B · 1000B | 40.0% | ~604 GB |
| 4 / 31 | GLM 5.2 · 753.3B | 36.7% | ~456 GB |
| 5 / 33 | DeepSeek V4 Pro 0813 · 1650.5B | 33.3% | ~991 GB |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- GLM 5.2, 753B, score 36.7% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4.1 Flash, 763B, score 64.4% — on the efficiency frontier (best score at its size or smaller).
MirrorCode: frequently asked questions
- What is the best open LLM on MirrorCode?
- DeepSeek V4.1 Flash is the top open model on MirrorCode, scoring 64.4%. Among all models tested — including proprietary ones — it ranks #11. The top model overall is Claude Sonnet 5.5 Max (Anthropic) at 97.8%.
- Can open models match proprietary models on MirrorCode?
- Not quite on MirrorCode: the strongest proprietary model (Claude Sonnet 5.5 Max) scores 97.8%, ahead of the best open model (DeepSeek V4.1 Flash) at 64.4% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.