llmrun
Coding

LiveBench Coding Leaderboard

LiveBench Coding evaluates code generation and completion on fresh, contamination-free programming tasks that are updated regularly.

Source: livebench19 open models ranked+51 proprietaryData through Jun 2026

Open modelsAll models

All models ranked on LiveBench Coding

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScoreVRAM @ Q4
1Claude Sonnet 5.5 Max · proprietary
91.4
API only
2Claude Opus 5.5 · proprietary
89.3
API only
3Claude Opus 5.5 Max · proprietary
89.3
API only
4Claude Sonnet 5.5 · proprietary
88.9
API only
5Claude Fable 5.1 Max · proprietary
86.4
API only
6Claude Fable 5 Max · proprietary
86.0
API only
7GPT 5.6 Sol Max · proprietary
83.9
API only
8GPT 5.2 Codex · proprietary
83.6
API only
9GPT 5.6 Luna Max · proprietary
82.9
API only
10Smaug Agentic · proprietary
82.5
API only
11GPT 5.5 · proprietary
82.2
API only
12Claude Opus 4.7 · proprietary
82.1
API only
13Claude Opus 4.8 Max · proprietary
81.8
API only
14GPT 6 Sol Max · proprietary
81.8
API only
15Claude Opus 5 Max · proprietary
81.5
API only
16Kimi K3 · 2779.9B
81.5
~1674 GB
17Muse Spark 1.3 · proprietary
81.1
API only
18GPT 6.1 Sol · proprietary
80.7
API only
19Claude Sonnet 5 · proprietary
80.7
API only
20Muse Wonder 1.0 Max · proprietary
80.7
API only
21GPT 6 Astra Max · proprietary
80.4
API only
22GPT 6.1 Sol Max · proprietary
80.4
API only
23DeepSeek V4.1 Flash · 763.2B
80.0
~458 GB
24Claude Opus 4.5 · proprietary
79.7
API only
25GLM 5.2 · 753.3B
79.7
~456 GB
26Claude Sonnet 4.6 · proprietary
79.3
API only
27GLM 5.3 · 753.3B
79.0
~456 GB
28GLM 5.3 Flash · 321.3B
79.0
~195 GB
29GPT 6 Luna Max · proprietary
79.0
API only
30Gemini 3.7 Flash · proprietary
78.9
API only
31Kimi K2.6 · 1026.9B
78.6
~620 GB
32GPT 5.6 Terra Max · proprietary
78.3
API only
33Claude Opus 4.6 · proprietary
78.2
API only
34Gemini 3.5 Flash · proprietary
78.2
API only
35Qwen3.6 Plus · proprietary
78.2
API only
36Claude Haiku 5.5 Max · proprietary
77.9
API only
37Gemini 3.6 Flash · proprietary
77.9
API only
38GPT 5.4 · proprietary
77.5
API only
39Muse Spark 1.2 · proprietary
77.5
API only
40DeepSeek V4 Pro 0813 · 1650.5B
77.2
~991 GB
41Grok 4.7 · proprietary
77.2
API only
42Mistral Large 4 · proprietary
77.2
API only
43Muse Spark 1.1 · proprietary
77.2
API only
44Smaug Flash · 304.2B
77.1
~609 GB
45Grok 4.6 · proprietary
76.8
API only
46Gemini 3.1 Pro · proprietary
76.5
API only
47Claude Haiku 5.5 · proprietary
76.4
API only
48Gemini 3.5 Flash Lite · proprietary
76.1
API only
49GPT 5.2 · proprietary
76.1
API only
50Ox Alpha Max · proprietary
75.8
API only
51Qwen3.8 27B · 27.8B
75.7
~17 GB
52Smaug Mini · proprietary
75.4
API only
53DeepSeek V4 Flash 0731 · 304.2B
75.0
~183 GB
54Qwen3.7 Max · proprietary
74.2
API only
55Kimi K2.7 Code · 1026.9B
74.0
~620 GB
56Qwen3.8 Max · proprietary
72.9
API only
57Qwen3.8 Flash Next · 180.0B
72.5
~108 GB
58Gemini 3.8 Flash · proprietary
72.5
API only
59Qwen3.6 27B · 27.8B
71.8
~17 GB
60GPT 5.4 Mini · proprietary
71.6
API only
61Inkling · 952.4B
71.0
~572 GB
62GPT 5.4 Nano · proprietary
70.8
API only
63NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 560.5B
70.7
~370 GB
64DeepSeek V4 Pro · 1598.8B
70.0
~960 GB
65Grok 4.3 · proprietary
69.9
API only
66DeepSeek V4 Flash · 290.9B
69.2
~175 GB
67Grok 4.5 · proprietary
68.6
API only
68DeepSeek V4 Flash Vision Exp · 304.6B
68.2
~183 GB
69MiniMax M3 · 427.0B
68.2
~257 GB
70Grok Build 0.1 · proprietary
65.4
API only

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →81.568.2GLM 5.3 · 753B · 79.0Kimi K2.6 · 1T · 78.6DeepSeek V4 Pro 0813 · 1.7T · 77.2DeepSeek V4 Flash 0731 · 304B · 75.0Kimi K2.7 Code · 1T · 74.0Qwen3.8 Flash Next · 180B · 72.5Qwen3.6 27B · 28B · 71.8Inkling · 952B · 71.0NVIDIA Nemotron 3 Ultra 550B A55B BF16 · 561B · 70.7DeepSeek V4 Pro · 1.6T · 70.0DeepSeek V4 Flash · 291B · 69.2DeepSeek V4 Flash Vision Exp · 305B · 68.2MiniMax M3 · 427B · 68.2Qwen3.8 27B · 28B · 75.7Qwen3.8 27BSmaug Flash · 304B · 77.1Smaug FlashGLM 5.3 Flash · 321B · 79.0GLM 5.2 · 753B · 79.7GLM 5.2DeepSeek V4.1 Flash · 763B · 80.0DeepSeek V4.1 FlashKimi K3 · 2.8T · 81.5Kimi K3
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Qwen3.8 27B, 28B, score 75.7 — on the efficiency frontier (best score at its size or smaller).
  • Smaug Flash, 304B, score 77.1 — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3 Flash, 321B, score 79.0 — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.2, 753B, score 79.7 — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4.1 Flash, 763B, score 80.0 — on the efficiency frontier (best score at its size or smaller).
  • Kimi K3, 2.8T, score 81.5 — on the efficiency frontier (best score at its size or smaller).

LiveBench Coding: frequently asked questions

What is the best open LLM on LiveBench Coding?
Kimi K3 is the top open model on LiveBench Coding, scoring 81.5. Among all models tested — including proprietary ones — it ranks #15. The top model overall is Claude Sonnet 5.5 Max (Anthropic) at 91.4.
What's the best LiveBench Coding model you can run on a 24 GB GPU?
Qwen3.8 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (~17 GB), scoring 75.7 on LiveBench Coding.
Can open models match proprietary models on LiveBench Coding?
Not quite on LiveBench Coding: the strongest proprietary model (Claude Sonnet 5.5 Max) scores 91.4, ahead of the best open model (Kimi K3) at 81.5 — but you can run the open one yourself.

Scores aggregated from livebench. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.