llmrun
Coding

FrontierCode Leaderboard

FrontierCode, from Cognition (the team behind Devin) with maintainers of open-source projects, gives an agent a real repository and issue and grades its patch on whether it's mergeable — held-out tests plus a maintainer-written rubric covering correctness, regression safety, style and project conventions. The Main set is the 100 hardest tasks.

Source: epoch11 open models ranked+25 proprietaryData through Sep 2026

Open modelsAll models

Open models ranked on FrontierCode

# shows rank among open models / rank overall (including proprietary).

#ModelScore
1 / 9Kimi K3 · 2779.9B
44.2%
2 / 17GLM 5.3 · 753.3B
40.1%
3 / 21GLM 5.3 Flash · 321.3B
31.8%
4 / 22Kimi K2.7 Code · 1026.9B
30.1%
5 / 23DeepSeek V4 Pro 0813 · 1650.5B
28.5%
6 / 27GLM 5.2 · 753.3B
24.5%
7 / 29DeepSeek V4 Flash 0731 · 304.2B
18.8%
8 / 30DeepSeek V4 Pro · 1598.8B
17.6%
9 / 31MiniMax M3 · 427.0B
14.7%
10 / 32Inkling · 952.4B
14.0%
11 / 33NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B
13.6%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

304B2.8Tmodel size (log scale) →44.2%13.6%Kimi K2.7 Code · 1T · 30.1%DeepSeek V4 Pro 0813 · 1.7T · 28.5%GLM 5.2 · 753B · 24.5%DeepSeek V4 Pro · 1.6T · 17.6%MiniMax M3 · 427B · 14.7%Inkling · 952B · 14.0%NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 561B · 13.6%DeepSeek V4 Flash 0731 · 304B · 18.8%DeepSeek V4 Flash 0731GLM 5.3 Flash · 321B · 31.8%GLM 5.3 FlashGLM 5.3 · 753B · 40.1%GLM 5.3Kimi K3 · 2.8T · 44.2%Kimi K3
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • DeepSeek V4 Flash 0731, 304B, score 18.8% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3 Flash, 321B, score 31.8% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3, 753B, score 40.1% — on the efficiency frontier (best score at its size or smaller).
  • Kimi K3, 2.8T, score 44.2% — on the efficiency frontier (best score at its size or smaller).

FrontierCode: frequently asked questions

What is the best open LLM on FrontierCode?
Kimi K3 is the top open model on FrontierCode, scoring 44.2%. Among all models tested — including proprietary ones — it ranks #9. The top model overall is Claude Fable 5 (unspecified) (Anthropic) at 53.5%.
Can open models match proprietary models on FrontierCode?
Not quite on FrontierCode: the strongest proprietary model (Claude Fable 5 (unspecified)) scores 53.5%, ahead of the best open model (Kimi K3) at 44.2% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.