llmrun
Reasoning

APEX-Agents Leaderboard

APEX-Agents (AI Productivity Index) is Mercor's evaluation of AI agents on long-horizon, multi-application professional-services work — investment-banking, management-consulting, and corporate-law style tasks — graded against expert-produced solutions. The score is Pass@1, the fraction solved correctly on the first attempt.

Source: epoch10 open models ranked+23 proprietaryData through Sep 2026

Open modelsAll models

Open models ranked on APEX-Agents

# shows rank among open models / rank overall (including proprietary).

#ModelScore
1 / 11GLM 5.3 · 753.3B
56.6%
2 / 15GLM 5.3 Flash · 321.3B
52.8%
3 / 18Kimi K3 · 2779.9B
50.6%
4 / 21DeepSeek V4 Pro 0813 · 1650.5B
47.3%
5 / 25GLM 5.1 · 753.9B
40.9%
6 / 26MiniMax M3 · 427.0B
37.7%
7 / 27Kimi K2.7 Code · 1026.9B
37.6%
8 / 30Inkling · 952.4B
33.8%
9 / 32Qwen3.5 397B A17B · 403.4B
24.9%
10 / 33GPT OSS 120B · 116.8B
4.4%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

117B2.8Tmodel size (log scale) →56.6%4.4%Kimi K3 · 2.8T · 50.6%DeepSeek V4 Pro 0813 · 1.7T · 47.3%GLM 5.1 · 754B · 40.9%MiniMax M3 · 427B · 37.7%Kimi K2.7 Code · 1T · 37.6%Inkling · 952B · 33.8%Qwen3.5 397B A17B · 403B · 24.9%GPT OSS 120B · 117B · 4.4%GPT OSS 120BGLM 5.3 Flash · 321B · 52.8%GLM 5.3 FlashGLM 5.3 · 753B · 56.6%GLM 5.3
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • GPT OSS 120B, 117B, score 4.4% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3 Flash, 321B, score 52.8% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3, 753B, score 56.6% — on the efficiency frontier (best score at its size or smaller).

APEX-Agents: frequently asked questions

What is the best open LLM on APEX-Agents?
GLM 5.3 is the top open model on APEX-Agents, scoring 56.6%. Among all models tested — including proprietary ones — it ranks #11. The top model overall is Claude Fable 5.1 (unspecified) (Anthropic) at 68.6%.
Can open models match proprietary models on APEX-Agents?
Not quite on APEX-Agents: the strongest proprietary model (Claude Fable 5.1 (unspecified)) scores 68.6%, ahead of the best open model (GLM 5.3) at 56.6% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.