llmrun
Coding

FrontierCode Leaderboard

FrontierCode, from Cognition (the team behind Devin) with maintainers of open-source projects, gives an agent a real repository and issue and grades its patch on whether it's mergeable — held-out tests plus a maintainer-written rubric covering correctness, regression safety, style and project conventions. The Main set is the 100 hardest tasks.

Source: epoch11 open models ranked+25 proprietaryData through Sep 2026

Open modelsAll models

All models ranked on FrontierCode

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScore
1Claude Fable 5 (unspecified) · proprietary
53.5%
2Claude Opus 5 Max · proprietary
53.4%
3GPT 6 Astra Max · proprietary
53.3%
4Claude Fable 5.1 (medium) · proprietary
50.9%
5Swe 2 Max · proprietary
50.0%
6Grok 4.6 (unspecified) · proprietary
48.0%
7GPT 5.6 Sol (unspecified) · proprietary
47.5%
8Claude Opus 4.8 (unspecified) · proprietary
46.5%
9Kimi K3 · 2779.9B
44.2%
10Gemini 3.7 Flash (unspecified) · proprietary
43.6%
11GPT 5.5 (unspecified) · proprietary
43.0%
12Claude Sonnet 5 (unspecified) · proprietary
42.7%
13Grok 4.5 (unspecified) · proprietary
42.4%
14Swe 1.7 · proprietary
42.0%
15GPT 5.6 Terra (unspecified) · proprietary
41.3%
16Gemini 3.8 Flash (medium) · proprietary
41.2%
17GLM 5.3 · 753.3B
40.1%
18GPT 5.6 Luna (unspecified) · proprietary
39.8%
19Claude Opus 4.7 (unspecified) · proprietary
38.5%
20Gemini 3.6 Flash (unspecified) · proprietary
34.4%
21GLM 5.3 Flash · 321.3B
31.8%
22Kimi K2.7 Code · 1026.9B
30.1%
23DeepSeek V4 Pro 0813 · 1650.5B
28.5%
24GPT 5.4 Mini (Mar 17, 2026, unspecified) · proprietary
27.0%
25Claude Opus 4.6 (unspecified) · proprietary
26.6%
26Composer 2.5 · proprietary
25.6%
27GLM 5.2 · 753.3B
24.5%
28Claude Sonnet 4.6 (unspecified) · proprietary
24.3%
29DeepSeek V4 Flash 0731 · 304.2B
18.8%
30DeepSeek V4 Pro · 1598.8B
17.6%
31MiniMax M3 · 427.0B
14.7%
32Inkling · 952.4B
14.0%
33NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B
13.6%
34Qwen3.7 Plus · proprietary
10.2%
35Swe 1.6 · proprietary
9.4%
36Mistral Medium 2604 · proprietary
8.0%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

304B2.8Tmodel size (log scale) →44.2%13.6%Kimi K2.7 Code · 1T · 30.1%DeepSeek V4 Pro 0813 · 1.7T · 28.5%GLM 5.2 · 753B · 24.5%DeepSeek V4 Pro · 1.6T · 17.6%MiniMax M3 · 427B · 14.7%Inkling · 952B · 14.0%NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 561B · 13.6%DeepSeek V4 Flash 0731 · 304B · 18.8%DeepSeek V4 Flash 0731GLM 5.3 Flash · 321B · 31.8%GLM 5.3 FlashGLM 5.3 · 753B · 40.1%GLM 5.3Kimi K3 · 2.8T · 44.2%Kimi K3
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • DeepSeek V4 Flash 0731, 304B, score 18.8% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3 Flash, 321B, score 31.8% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3, 753B, score 40.1% — on the efficiency frontier (best score at its size or smaller).
  • Kimi K3, 2.8T, score 44.2% — on the efficiency frontier (best score at its size or smaller).

FrontierCode: frequently asked questions

What is the best open LLM on FrontierCode?
Kimi K3 is the top open model on FrontierCode, scoring 44.2%. Among all models tested — including proprietary ones — it ranks #9. The top model overall is Claude Fable 5 (unspecified) (Anthropic) at 53.5%.
Can open models match proprietary models on FrontierCode?
Not quite on FrontierCode: the strongest proprietary model (Claude Fable 5 (unspecified)) scores 53.5%, ahead of the best open model (Kimi K3) at 44.2% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.