llmrun
Coding

SWE-bench Verified Leaderboard

SWE-bench Verified tests whether a model can resolve real GitHub issues from popular open-source Python projects, scored on the official swebench.com leaderboard as the percentage of human-validated issues actually fixed. It is the headline measure of practical, agentic software-engineering ability — where open-weight models like Qwen3-Coder, GLM-4.6, Kimi K2 and DeepSWE are now competitive with the frontier.

Source: swebench13 open models ranked+150 proprietaryData through Feb 2026

Open modelsAll models

Open models ranked on SWE-bench Verified

# shows rank among open models / rank overall (including proprietary).

#ModelScore
1 / 33Kimi K2 Instruct 0905 · 1026.5B
71.2%
2 / 47Qwen3 Coder 480B A35B Instruct · 480.2B
69.6%
3 / 49GLM 4.6 · 356.8B
68.2%
4 / 60Kimi K2 Instruct · 1026.5B
65.4%
5 / 65GLM 4.5 · 358.3B
64.2%
6 / 67Kimi K2 Thinking · 1058.1B
63.4%
7 / 73MiniMax M2 · 228.7B
61.0%
8 / 75Qwen3 Coder 30B A3B Instruct · 30.5B
60.4%
9 / 77DeepSeek V3.2 · 685.4B
60.0%
10 / 79DeepSWE Preview · 32.8B
58.8%
11 / 105Qwen2.5 Coder 32B Instruct · 32.8B
47.0%
12 / 117DeepSeek v3 0324 · 684.5B
42.0%
13 / 143GPT OSS 120B · 120.4B
26.0%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →71.2%26.0%Kimi K2 Instruct · 1T · 65.4%GLM 4.5 · 358B · 64.2%Kimi K2 Thinking · 1.1T · 63.4%DeepSeek V3.2 · 685B · 60.0%DeepSWE Preview · 33B · 58.8%Qwen2.5 Coder 32B Instruct · 33B · 47.0%DeepSeek v3 0324 · 685B · 42.0%GPT OSS 120B · 120B · 26.0%Qwen3 Coder 30B A3B Instruct · 31B · 60.4%Qwen3 Coder 30B A3B I…MiniMax M2 · 229B · 61.0%MiniMax M2GLM 4.6 · 357B · 68.2%GLM 4.6Qwen3 Coder 480B A35B Instruct · 480B · 69.6%Qwen3 Coder 480B A35B…Kimi K2 Instruct 0905 · 1T · 71.2%Kimi K2 Instruct 0905
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Qwen3 Coder 30B A3B Instruct, 31B, score 60.4% — on the efficiency frontier (best score at its size or smaller).
  • MiniMax M2, 229B, score 61.0% — on the efficiency frontier (best score at its size or smaller).
  • GLM 4.6, 357B, score 68.2% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3 Coder 480B A35B Instruct, 480B, score 69.6% — on the efficiency frontier (best score at its size or smaller).
  • Kimi K2 Instruct 0905, 1T, score 71.2% — on the efficiency frontier (best score at its size or smaller).

SWE-bench Verified: frequently asked questions

What is the best open LLM on SWE-bench Verified?
Kimi K2 Instruct 0905 is the top open model on SWE-bench Verified, scoring 71.2%. Among all models tested — including proprietary ones — it ranks #32. The top model overall is live-SWE-agent + Claude 4.5 Opus medium (20251101) at 79.2%.
What's the best SWE-bench Verified model you can run on a 24 GB GPU?
Qwen3 Coder 30B A3B Instruct is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 17 GB), scoring 60.4% on SWE-bench Verified.
Can open models match proprietary models on SWE-bench Verified?
Not quite on SWE-bench Verified: the strongest proprietary model (live-SWE-agent + Claude 4.5 Opus medium (20251101)) scores 79.2%, ahead of the best open model (Kimi K2 Instruct 0905) at 71.2% — but you can run the open one yourself.

Scores aggregated from swebench. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.