Coding
FrontierCode Leaderboard
FrontierCode, from Cognition (the team behind Devin) with maintainers of open-source projects, gives an agent a real repository and issue and grades its patch on whether it's mergeable — held-out tests plus a maintainer-written rubric covering correctness, regression safety, style and project conventions. The Main set is the 100 hardest tasks.
Source: epoch14 open models ranked+32 proprietaryData through Oct 2026
Open models ranked on FrontierCode
# shows rank among open models / rank overall (including proprietary).
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 / 16 | Kimi K3 · 2779.9B | 44.2% | ~1674 GB |
| 2 / 17 | DeepSeek V4.1 Flash · 763.2B | 43.9% | ~458 GB |
| 3 / 26 | GLM 5.3 · 753.3B | 40.1% | ~456 GB |
| 4 / 28 | Mistral Large 4 1T A52B · 1000B | 39.4% | ~604 GB |
| 5 / 31 | GLM 5.3 Flash · 321.3B | 31.8% | ~195 GB |
| 6 / 32 | Kimi K2.7 Code · 1026.9B | 30.1% | ~620 GB |
| 7 / 33 | DeepSeek V4 Pro 0813 · 1650.5B | 28.5% | ~991 GB |
| 8 / 37 | GLM 5.2 · 753.3B | 24.5% | ~456 GB |
| 9 / 39 | DeepSeek V4 Flash 0731 · 304.2B | 18.8% | ~183 GB |
| 10 / 40 | DeepSeek V4 Pro · 1598.8B | 17.6% | ~960 GB |
| 11 / 41 | MiniMax M3 · 427.0B | 14.7% | ~257 GB |
| 12 / 42 | Inkling · 952.4B | 14.0% | ~572 GB |
| 13 / 43 | NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B | 13.6% | ~370 GB |
| 14 / 46 | Mistral Medium 3.5 128B · 127.7B | 8.0% | ~78 GB |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- Mistral Medium 3.5 128B, 128B, score 8.0% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Flash 0731, 304B, score 18.8% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.3 Flash, 321B, score 31.8% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.3, 753B, score 40.1% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4.1 Flash, 763B, score 43.9% — on the efficiency frontier (best score at its size or smaller).
- Kimi K3, 2.8T, score 44.2% — on the efficiency frontier (best score at its size or smaller).
FrontierCode: frequently asked questions
- What is the best open LLM on FrontierCode?
- Kimi K3 is the top open model on FrontierCode, scoring 44.2%. Among all models tested — including proprietary ones — it ranks #16. The top model overall is Claude Opus 5.5 (medium) (Anthropic) at 54.6%.
- Can open models match proprietary models on FrontierCode?
- Not quite on FrontierCode: the strongest proprietary model (Claude Opus 5.5 (medium)) scores 54.6%, ahead of the best open model (Kimi K3) at 44.2% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.