llmrun
Reasoning

APEX-Agents Leaderboard

APEX-Agents (AI Productivity Index) is Mercor's evaluation of AI agents on long-horizon, multi-application professional-services work — investment-banking, management-consulting, and corporate-law style tasks — graded against expert-produced solutions. The score is Pass@1, the fraction solved correctly on the first attempt.

Source: epoch10 open models ranked+23 proprietaryData through Sep 2026

Open modelsAll models

All models ranked on APEX-Agents

Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.

#ModelScore
1Claude Fable 5.1 (unspecified) · proprietary
68.6%
2Gemini 3.7 Flash (unspecified) · proprietary
67.8%
3Claude Opus 5 Max · proprietary
65.8%
4Grok 4.6 (unspecified) · proprietary
65.3%
5GPT 6 Astra (unspecified) · proprietary
64.7%
6Gemini 3.8 Flash (unspecified) · proprietary
64.3%
7Claude Fable 5 · proprietary
63.6%
8Claude Fable 5.1 (high) · proprietary
59.7%
9GPT 5.6 Terra Max · proprietary
58.2%
10Muse Spark 1.3 (unspecified) · proprietary
57.8%
11GLM 5.3 · 753.3B
56.6%
12Grok 4.5 (unspecified) · proprietary
56.2%
13GPT 5.5 (unspecified) · proprietary
55.1%
14Claude Sonnet 5 (unspecified) · proprietary
54.5%
15GLM 5.3 Flash · 321.3B
52.8%
16GPT 5.4 (Mar 05, 2026, unspecified) · proprietary
52.4%
17GPT 5.6 Sol Promax · proprietary
51.4%
18Kimi K3 · 2779.9B
50.6%
19Claude Opus 4.7 Max · proprietary
49.2%
20Claude Opus 4.8 Max · proprietary
48.9%
21DeepSeek V4 Pro 0813 · 1650.5B
47.3%
22Gemini 3.6 Flash (unspecified) · proprietary
46.9%
23Claude Opus 4.6 Max · proprietary
46.3%
24Claude Sonnet 4.6 (high) · proprietary
43.0%
25GLM 5.1 · 753.9B
40.9%
26MiniMax M3 · 427.0B
37.7%
27Kimi K2.7 Code · 1026.9B
37.6%
28Muse Spark 1.2 · proprietary
36.4%
29Gemini 3.1 Pro Preview · proprietary
35.3%
30Inkling · 952.4B
33.8%
31Gemini 3.5 Flash (unspecified) · proprietary
27.5%
32Qwen3.5 397B A17B · 403.4B
24.9%
33GPT OSS 120B · 116.8B
4.4%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

117B2.8Tmodel size (log scale) →56.6%4.4%Kimi K3 · 2.8T · 50.6%DeepSeek V4 Pro 0813 · 1.7T · 47.3%GLM 5.1 · 754B · 40.9%MiniMax M3 · 427B · 37.7%Kimi K2.7 Code · 1T · 37.6%Inkling · 952B · 33.8%Qwen3.5 397B A17B · 403B · 24.9%GPT OSS 120B · 117B · 4.4%GPT OSS 120BGLM 5.3 Flash · 321B · 52.8%GLM 5.3 FlashGLM 5.3 · 753B · 56.6%GLM 5.3
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • GPT OSS 120B, 117B, score 4.4% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3 Flash, 321B, score 52.8% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3, 753B, score 56.6% — on the efficiency frontier (best score at its size or smaller).

APEX-Agents: frequently asked questions

What is the best open LLM on APEX-Agents?
GLM 5.3 is the top open model on APEX-Agents, scoring 56.6%. Among all models tested — including proprietary ones — it ranks #11. The top model overall is Claude Fable 5.1 (unspecified) (Anthropic) at 68.6%.
Can open models match proprietary models on APEX-Agents?
Not quite on APEX-Agents: the strongest proprietary model (Claude Fable 5.1 (unspecified)) scores 68.6%, ahead of the best open model (GLM 5.3) at 56.6% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.