Coding
FrontierCode Leaderboard
FrontierCode, from Cognition (the team behind Devin) with maintainers of open-source projects, gives an agent a real repository and issue and grades its patch on whether it's mergeable — held-out tests plus a maintainer-written rubric covering correctness, regression safety, style and project conventions. The Main set is the 100 hardest tasks.
Source: epoch14 open models ranked+32 proprietaryData through Oct 2026
All models ranked on FrontierCode
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 | Claude Opus 5.5 (medium) · proprietary | 54.6% | API only |
| 2 | Claude Fable 5 (unspecified) · proprietary | 53.5% | API only |
| 3 | Claude Opus 5 Max · proprietary | 53.4% | API only |
| 4 | GPT 6 Astra Max · proprietary | 53.3% | API only |
| 5 | Claude Sonnet 5.5 (xhigh) · proprietary | 52.1% | API only |
| 6 | Claude Fable 5.1 (medium) · proprietary | 50.9% | API only |
| 7 | GPT 6.1 Sol (medium) · proprietary | 50.2% | API only |
| 8 | Swe 2 Max · proprietary | 50.0% | API only |
| 9 | GPT 6 Sol Max · proprietary | 49.3% | API only |
| 10 | Grok 4.6 (unspecified) · proprietary | 48.0% | API only |
| 11 | Grok 4.7 (unspecified) · proprietary | 47.6% | API only |
| 12 | Gemini 4 Argon (medium) · proprietary | 47.5% | API only |
| 13 | GPT 5.6 Sol (unspecified) · proprietary | 47.5% | API only |
| 14 | Claude Opus 4.8 (unspecified) · proprietary | 46.5% | API only |
| 15 | Claude Haiku 5.5 Max · proprietary | 46.4% | API only |
| 16 | Kimi K3 · 2779.9B | 44.2% | ~1674 GB |
| 17 | DeepSeek V4.1 Flash · 763.2B | 43.9% | ~458 GB |
| 18 | Gemini 3.7 Flash (unspecified) · proprietary | 43.6% | API only |
| 19 | GPT 5.5 (unspecified) · proprietary | 43.0% | API only |
| 20 | Claude Sonnet 5 (unspecified) · proprietary | 42.7% | API only |
| 21 | Grok 4.5 (unspecified) · proprietary | 42.4% | API only |
| 22 | GPT 6 Luna Max · proprietary | 42.4% | API only |
| 23 | Swe 1.7 · proprietary | 42.0% | API only |
| 24 | GPT 5.6 Terra (unspecified) · proprietary | 41.3% | API only |
| 25 | Gemini 3.8 Flash (medium) · proprietary | 41.2% | API only |
| 26 | GLM 5.3 · 753.3B | 40.1% | ~456 GB |
| 27 | GPT 5.6 Luna (unspecified) · proprietary | 39.8% | API only |
| 28 | Mistral Large 4 1T A52B · 1000B | 39.4% | ~604 GB |
| 29 | Claude Opus 4.7 (unspecified) · proprietary | 38.5% | API only |
| 30 | Gemini 3.6 Flash (unspecified) · proprietary | 34.4% | API only |
| 31 | GLM 5.3 Flash · 321.3B | 31.8% | ~195 GB |
| 32 | Kimi K2.7 Code · 1026.9B | 30.1% | ~620 GB |
| 33 | DeepSeek V4 Pro 0813 · 1650.5B | 28.5% | ~991 GB |
| 34 | GPT 5.4 Mini (Mar 17, 2026, unspecified) · proprietary | 27.0% | API only |
| 35 | Claude Opus 4.6 (unspecified) · proprietary | 26.6% | API only |
| 36 | Composer 2.5 · proprietary | 25.6% | API only |
| 37 | GLM 5.2 · 753.3B | 24.5% | ~456 GB |
| 38 | Claude Sonnet 4.6 (unspecified) · proprietary | 24.3% | API only |
| 39 | DeepSeek V4 Flash 0731 · 304.2B | 18.8% | ~183 GB |
| 40 | DeepSeek V4 Pro · 1598.8B | 17.6% | ~960 GB |
| 41 | MiniMax M3 · 427.0B | 14.7% | ~257 GB |
| 42 | Inkling · 952.4B | 14.0% | ~572 GB |
| 43 | NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B | 13.6% | ~370 GB |
| 44 | Qwen3.7 Plus · proprietary | 10.2% | API only |
| 45 | Swe 1.6 · proprietary | 9.4% | API only |
| 46 | Mistral Medium 3.5 128B · 127.7B | 8.0% | ~78 GB |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- Mistral Medium 3.5 128B, 128B, score 8.0% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4 Flash 0731, 304B, score 18.8% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.3 Flash, 321B, score 31.8% — on the efficiency frontier (best score at its size or smaller).
- GLM 5.3, 753B, score 40.1% — on the efficiency frontier (best score at its size or smaller).
- DeepSeek V4.1 Flash, 763B, score 43.9% — on the efficiency frontier (best score at its size or smaller).
- Kimi K3, 2.8T, score 44.2% — on the efficiency frontier (best score at its size or smaller).
FrontierCode: frequently asked questions
- What is the best open LLM on FrontierCode?
- Kimi K3 is the top open model on FrontierCode, scoring 44.2%. Among all models tested — including proprietary ones — it ranks #16. The top model overall is Claude Opus 5.5 (medium) (Anthropic) at 54.6%.
- Can open models match proprietary models on FrontierCode?
- Not quite on FrontierCode: the strongest proprietary model (Claude Opus 5.5 (medium)) scores 54.6%, ahead of the best open model (Kimi K3) at 44.2% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.