Coding
SWE-bench Verified Leaderboard
SWE-bench Verified tests whether a model can resolve real GitHub issues from popular open-source Python projects, scored on the official swebench.com leaderboard as the percentage of human-validated issues actually fixed. It is the headline measure of practical, agentic software-engineering ability — where open-weight models like Qwen3-Coder, GLM-4.6, Kimi K2 and DeepSWE are now competitive with the frontier.
Source: swebench13 open models ranked+150 proprietaryData through Feb 2026
All models ranked on SWE-bench Verified
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score |
|---|---|---|
| 1 | live-SWE-agent + Claude 4.5 Opus medium (20251101) · proprietary | 79.2% |
| 2 | Sonar Foundation Agent + Claude 4.5 Opus · proprietary | 79.2% |
| 3 | TRAE + Doubao-Seed-Code · proprietary | 78.8% |
| 4 | live-SWE-agent + Gemini 3 Pro Preview (2025-11-18) · proprietary | 77.4% |
| 5 | Atlassian Rovo Dev (2025-09-02) · proprietary | 76.8% |
| 6 | EPAM AI/Run Developer Agent v20250719 + Claude 4 Sonnet · proprietary | 76.8% |
| 7 | mini-SWE-agent + Claude 4.5 Opus (high reasoning) · proprietary | 76.8% |
| 8 | ACoder · proprietary | 76.4% |
| 9 | mini-SWE-agent + Gemini 3 Flash (high reasoning) · proprietary | 75.8% |
| 10 | mini-SWE-agent + MiniMax M2.5 (high reasoning) · proprietary | 75.8% |
| 11 | mini-SWE-agent + Claude Opus 4.6 · proprietary | 75.6% |
| 12 | Warp · proprietary | 75.6% |
| 13 | TRAE + Claude Sonnet 4 + Opus 4 + Sonnet 3.7 + Gemini 2.5 Pro · proprietary | 75.2% |
| 14 | Harness AI · proprietary | 74.8% |
| 15 | Sonar Foundation Agent + Claude 4.5 Sonnet · proprietary | 74.8% |
| 16 | JoyCode + Claude 4 Sonnet + GPT-4.1 · proprietary | 74.6% |
| 17 | Lingxi-v1.5_claude-4-sonnet-20250514 · proprietary | 74.6% |
| 18 | mini-SWE-agent + Claude 4.5 Opus medium (20251101) · proprietary | 74.4% |
| 19 | Prometheus-v1.2.1 + GPT-5 · proprietary | 74.4% |
| 20 | Refact.ai Agent + Claude 4 Sonnet + o4-mini · proprietary | 74.4% |
| 21 | mini-SWE-agent + Gemini 3 Pro Preview (2025-11-18) · proprietary | 74.2% |
| 22 | Salesforce AI Research SAGE (OpenHands) · proprietary | 73.8% |
| 23 | Tools + Claude 4 Opus (2025-05-22) · proprietary | 73.2% |
| 24 | Salesforce AI Research SAGE (bash-only) · proprietary | 73.0% |
| 25 | mini-SWE-agent + GLM-5 (high reasoning) · proprietary | 72.8% |
| 26 | mini-SWE-agent + GPT-5-2 (high reasoning) · proprietary | 72.8% |
| 27 | mini-SWE-agent + GPT-5-2 Codex · proprietary | 72.8% |
| 28 | Tools + Claude 4 Sonnet (2025-05-22) · proprietary | 72.4% |
| 29 | mini-SWE-agent + GPT-5.2 (2025-12-11) (high reasoning) · proprietary | 71.8% |
| 30 | OpenHands + GPT-5 · proprietary | 71.8% |
| 31 | mini-SWE-agent + Claude 4.5 Sonnet (high reasoning) · proprietary | 71.4% |
| 32 | Bloop · proprietary | 71.2% |
| 33 | Kimi K2 Instruct 0905 · 1026.5B | 71.2% |
| 34 | Prometheus-v1.2 + GPT-5 · proprietary | 71.2% |
| 35 | Qodo Command · proprietary | 71.2% |
| 36 | mini-SWE-agent + Kimi K2.5 (high reasoning) · proprietary | 70.8% |
| 37 | Moatless Tools + Claude 4 Sonnet · proprietary | 70.8% |
| 38 | mini-SWE-agent + Claude 4.5 Sonnet (20250929) · proprietary | 70.6% |
| 39 | TRAE · proprietary | 70.6% |
| 40 | Augment Agent v1 · proprietary | 70.4% |
| 41 | OpenHands + Claude 4 Sonnet · proprietary | 70.4% |
| 42 | Refact.ai Agent · proprietary | 70.4% |
| 43 | devlo · proprietary | 70.2% |
| 44 | mini-SWE-agent + DeepSeek V3.2 (high reasoning) · proprietary | 70.0% |
| 45 | Zencoder (2025-04-30) · proprietary | 70.0% |
| 46 | mini-SWE-agent + Gemini 3 Pro · proprietary | 69.6% |
| 47 | Qwen3 Coder 480B A35B Instruct · 480.2B | 69.6% |
| 48 | mini-SWE-agent + GPT-5.2 (2025-12-11) · proprietary | 69.0% |
| 49 | GLM 4.6 · 356.8B | 68.2% |
| 50 | Nemotron-CORTEXA · proprietary | 68.2% |
| 51 | mini-SWE-agent + Claude 4 Opus (20250514) · proprietary | 67.6% |
| 52 | mini-SWE-agent + Claude 4.5 Haiku (high reasoning) · proprietary | 66.6% |
| 53 | SWE-agent + Claude 4 Sonnet · proprietary | 66.6% |
| 54 | Aime-coder v1 + Anthopic Claude 3.7 Sonnet · proprietary | 66.4% |
| 55 | mini-SWE-agent + GPT-5.1 (2025-11-13) (medium reasoning) · proprietary | 66.0% |
| 56 | mini-SWE-agent + GPT-5.1-codex (medium reasoning) · proprietary | 66.0% |
| 57 | OpenHands · proprietary | 65.8% |
| 58 | Amazon Q Developer Agent (v20250405-dev) · proprietary | 65.4% |
| 59 | Augment Agent v0 · proprietary | 65.4% |
| 60 | Kimi K2 Instruct · 1026.5B | 65.4% |
| 61 | mini-SWE-agent + GPT-5 (2025-08-07) (medium reasoning) · proprietary | 65.0% |
| 62 | mini-SWE-agent + Claude 4 Sonnet (20250514) · proprietary | 64.9% |
| 63 | PatchPilot-v1.1 · proprietary | 64.6% |
| 64 | W&B Programmer O1 crosscheck5 · proprietary | 64.6% |
| 65 | GLM 4.5 · 358.3B | 64.2% |
| 66 | AgentScope · proprietary | 63.4% |
| 67 | Kimi K2 Thinking · 1058.1B | 63.4% |
| 68 | Tools + Claude 3.7 Sonnet (2025-02-24) · proprietary | 63.2% |
| 69 | Blackbox AI Agent · proprietary | 62.8% |
| 70 | EPAM AI/Run Developer Agent v20250219 + Anthopic Claude 3.5 Sonnet · proprietary | 62.8% |
| 71 | SWE-agent + Claude 3.7 Sonnet w/ Review Heavy · proprietary | 62.4% |
| 72 | CodeStory Midwit Agent + swe-search · proprietary | 62.2% |
| 73 | MiniMax M2 · 228.7B | 61.0% |
| 74 | OpenHands + 4x Scaled (2024-02-03) · proprietary | 60.8% |
| 75 | Qwen3 Coder 30B A3B Instruct · 30.5B | 60.4% |
| 76 | Learn-by-interact · proprietary | 60.2% |
| 77 | DeepSeek V3.2 · 685.4B | 60.0% |
| 78 | mini-SWE-agent + GPT-5 mini (2025-08-07) (medium reasoning) · proprietary | 59.8% |
| 79 | DeepSWE Preview · 32.8B | 58.8% |
| 80 | mini-SWE-agent + o3 (2025-04-16) · proprietary | 58.4% |
| 81 | Emergent E1 (v2024-12-23) · proprietary | 57.2% |
| 82 | Artemis Agent v2 (2025-09-24) · proprietary | 57.0% |
| 83 | Gru(2024-12-08) · proprietary | 57.0% |
| 84 | SWE-Rizzo · proprietary | 56.6% |
| 85 | devstral-small-2512 · proprietary | 56.4% |
| 86 | mini-SWE-agent + GPT-5 Mini · proprietary | 56.2% |
| 87 | EPAM AI/Run Developer Agent v20241212 + Anthopic Claude 3.5 Sonnet · proprietary | 55.4% |
| 88 | Amazon Q Developer Agent (v20241202-dev) · proprietary | 55.0% |
| 89 | devstral-2512 · proprietary | 53.8% |
| 90 | FrogBoss-32B-2510 · proprietary | 53.6% |
| 91 | mini-SWE-agent + Gemini 2.5 Pro (2025-05-06) · proprietary | 53.6% |
| 92 | Bracket.sh · proprietary | 53.2% |
| 93 | OpenHands + CodeAct v2.1 (claude-3-5-sonnet-20241022) · proprietary | 53.0% |
| 94 | mini-SWE-agent + Claude 3.7 Sonnet (20250219) · proprietary | 52.8% |
| 95 | Google Jules + Gemini 2.0 Flash (v20241212-experimental) · proprietary | 52.2% |
| 96 | Engine Labs (2024-11-25) · proprietary | 51.8% |
| 97 | AutoCodeRover-v2.1 (Claude-3.5-Sonnet-20241022) · proprietary | 51.6% |
| 98 | Agentless-1.5 + Claude-3.5 Sonnet (20241022) · proprietary | 50.8% |
| 99 | Bytedance MarsCode Agent · proprietary | 50.0% |
| 100 | Solver (2024-10-28) · proprietary | 50.0% |
| 101 | nFactorial (2024-11-05) · proprietary | 49.2% |
| 102 | Tools + Claude 3.5 Sonnet (2024-10-22) · proprietary | 49.0% |
| 103 | Composio SWE-Kit (2024-10-25) · proprietary | 48.6% |
| 104 | AppMap Navie v2 · proprietary | 47.2% |
| 105 | Qwen2.5 Coder 32B Instruct · 32.8B | 47.0% |
| 106 | Devstral-Small-2505 · proprietary | 46.8% |
| 107 | Emergent E1 (v2024-10-12) · proprietary | 46.6% |
| 108 | AutoCodeRover-v2.0 (Claude-3.5-Sonnet-20241022) · proprietary | 46.2% |
| 109 | PatchPilot + Co-PatcheR · proprietary | 46.0% |
| 110 | Solver (2024-09-12) · proprietary | 45.4% |
| 111 | Gru(2024-08-24) · proprietary | 45.2% |
| 112 | FrogMini-14B-2510 · proprietary | 45.0% |
| 113 | mini-SWE-agent + o4-mini (2025-04-16) · proprietary | 45.0% |
| 114 | CodeShellAgent + Gemini 2.0 Flash (Experimental) · proprietary | 44.2% |
| 115 | Agentless Lite + O3 Mini (20250214) · proprietary | 42.4% |
| 116 | Amazon Nova Premier 1.0 (2025-04-30) · proprietary | 42.4% |
| 117 | DeepSeek v3 0324 · 684.5B | 42.0% |
| 118 | nFactorial (2024-10-30) · proprietary | 41.6% |
| 119 | ugaiforge · proprietary | 41.6% |
| 120 | SWE-RL (Llama3-SWE-RL-70B + Agentless Mini) (20250226) · proprietary | 41.2% |
| 121 | Composio SWEkit + Claude 3.5 Sonnet (2024-10-16) · proprietary | 40.6% |
| 122 | Honeycomb · proprietary | 40.6% |
| 123 | Nebius AI Qwen 2.5 72B Generator + LLama 3.1 70B Critic · proprietary | 40.6% |
| 124 | Tools + Claude 3.5 Haiku · proprietary | 40.6% |
| 125 | Qwen 2.5 · proprietary | 40.2% |
| 126 | EPAM AI/Run Developer Agent v20241029 + Anthopic Claude 3.5 Sonnet · proprietary | 39.6% |
| 127 | mini-SWE-agent + GPT-4.1 (2025-04-14) · proprietary | 39.6% |
| 128 | Agentless-1.5 + GPT 4o (2024-05-13) · proprietary | 38.8% |
| 129 | Amazon Q Developer Agent (v20240719-dev) · proprietary | 38.8% |
| 130 | AutoCodeRover (v20240620) + GPT 4o (2024-05-13) · proprietary | 38.4% |
| 131 | devstral-small-2507 · proprietary | 38.0% |
| 132 | Factory Code Droid · proprietary | 37.0% |
| 133 | mini-SWE-agent + GPT-5 nano (2025-08-07) (medium reasoning) · proprietary | 34.8% |
| 134 | SWE-agent + Claude 3.5 Sonnet · proprietary | 33.6% |
| 135 | MASAI + GPT 4o (2024-06-12) · proprietary | 32.6% |
| 136 | Artemis Agent v1 (2024-11-20) · proprietary | 32.0% |
| 137 | nFactorial (2024-10-07) · proprietary | 31.6% |
| 138 | SWE-Fixer (Qwen2.5-7b retriever + Qwen2.5-72b editor) 20241128 · proprietary | 30.2% |
| 139 | Lingma Agent + Lingma SWE-GPT 72b (v0925) · proprietary | 28.8% |
| 140 | mini-SWE-agent + Gemini 2.5 Flash (2025-04-17) · proprietary | 28.7% |
| 141 | EPAM AI/Run Developer Agent + GPT4o · proprietary | 27.0% |
| 142 | AppMap Navie + GPT 4o (2024-05-13) · proprietary | 26.2% |
| 143 | GPT OSS 120B · 120.4B | 26.0% |
| 144 | nFactorial (2024-10-01) · proprietary | 25.8% |
| 145 | Amazon Q Developer Agent (v20240430-dev) · proprietary | 25.6% |
| 146 | Lingma Agent + Lingma SWE-GPT 72b (v0918) · proprietary | 25.0% |
| 147 | mini-SWE-agent + GPT-4.1-mini (2025-04-14) · proprietary | 23.9% |
| 148 | MCTS-Refine-7B · proprietary | 23.2% |
| 149 | SWE-agent + GPT 4o (2024-05-13) · proprietary | 23.2% |
| 150 | SWE-agent + GPT 4 (1106) · proprietary | 22.4% |
| 151 | mini-SWE-agent + GPT-4o (2024-11-20) · proprietary | 21.6% |
| 152 | mini-SWE-agent + Llama 4 Maverick Instruct · proprietary | 21.0% |
| 153 | Lingma Agent + Lingma SWE-GPT 7b (v0925) · proprietary | 18.2% |
| 154 | SWE-agent + Claude 3 Opus · proprietary | 15.8% |
| 155 | mini-SWE-agent + Gemini 2.0 flash · proprietary | 13.5% |
| 156 | Lingma Agent + Lingma SWE-GPT 7b (v0918) · proprietary | 10.2% |
| 157 | mini-SWE-agent + Llama 4 Scout Instruct · proprietary | 9.1% |
| 158 | RAG + Claude 3 Opus · proprietary | 7.0% |
| 159 | RAG + Claude 2 · proprietary | 4.4% |
| 160 | RAG + GPT 4 (1106) · proprietary | 2.8% |
| 161 | RAG + SWE-Llama 7B · proprietary | 1.4% |
| 162 | RAG + SWE-Llama 13B · proprietary | 1.2% |
| 163 | RAG + ChatGPT 3.5 · proprietary | 0.4% |
Score vs model size
Which models give the most quality for their size — the ones worth running locally.
- Qwen3 Coder 30B A3B Instruct, 31B, score 60.4% — on the efficiency frontier (best score at its size or smaller).
- MiniMax M2, 229B, score 61.0% — on the efficiency frontier (best score at its size or smaller).
- GLM 4.6, 357B, score 68.2% — on the efficiency frontier (best score at its size or smaller).
- Qwen3 Coder 480B A35B Instruct, 480B, score 69.6% — on the efficiency frontier (best score at its size or smaller).
- Kimi K2 Instruct 0905, 1T, score 71.2% — on the efficiency frontier (best score at its size or smaller).
SWE-bench Verified: frequently asked questions
- What is the best open LLM on SWE-bench Verified?
- Kimi K2 Instruct 0905 is the top open model on SWE-bench Verified, scoring 71.2%. Among all models tested — including proprietary ones — it ranks #32. The top model overall is live-SWE-agent + Claude 4.5 Opus medium (20251101) at 79.2%.
- What's the best SWE-bench Verified model you can run on a 24 GB GPU?
- Qwen3 Coder 30B A3B Instruct is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 17 GB), scoring 60.4% on SWE-bench Verified.
- Can open models match proprietary models on SWE-bench Verified?
- Not quite on SWE-bench Verified: the strongest proprietary model (live-SWE-agent + Claude 4.5 Opus medium (20251101)) scores 79.2%, ahead of the best open model (Kimi K2 Instruct 0905) at 71.2% — but you can run the open one yourself.
Scores aggregated from swebench. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.