What is the best open LLM on Vending-Bench 2?

GLM 5.2 is the top open model on Vending-Bench 2, scoring 8313.8. Among all models tested — including proprietary ones — it ranks #7. The top model overall is GPT 6 Astra (unspecified) (OpenAI) at 15514.7.

What's the best Vending-Bench 2 model you can run on a 24 GB GPU?

Qwen3.5 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 15 GB), scoring 202.0 on Vending-Bench 2.

Can open models match proprietary models on Vending-Bench 2?

Not quite on Vending-Bench 2: the strongest proprietary model (GPT 6 Astra (unspecified)) scores 15514.7, ahead of the best open model (GLM 5.2) at 8313.8 — but you can run the open one yourself.

Reasoning

Vending-Bench 2 Leaderboard

Name: Vending-Bench 2 — open LLM scores
Creator: epoch

Vending-Bench 2, from Andon Labs, is a long-horizon agent benchmark in which the model runs a simulated vending-machine business for a year, handling suppliers, pricing and cash flow. The score is the final money balance in US dollars.

Source: epoch17 open models ranked+46 proprietaryData through Sep 2026

Open models All models

Open models ranked on Vending-Bench 2

# shows rank among open models / rank overall (including proprietary).

#	Model	Score
1 / 7	GLM 5.2 · 753.3B	8313.8
2 / 8	GLM 5.3 · 753.3B	8163.6
3 / 15	Kimi K2.6 · 1026.9B	6204.6
4 / 20	GLM 5.1 · 753.9B	5634.4
5 / 24	Kimi K3 · 2779.9B	5165.0
6 / 27	Kimi K2.7 Code · 1026.9B	5082.9
7 / 33	GLM 5 · 753.9B	4432.1
8 / 42	DeepSeek V4 Pro · 1598.8B	3284.5
9 / 44	GLM 4.7 · 358.3B	2376.8
10 / 45	MiniMax M3 · 427.0B	2157.8
11 / 47	Kimi K2.5 · 1026.9B	1198.5
12 / 49	DeepSeek V3.2 Exp · 685.4B	1034.0
13 / 55	Qwen3.5 27B · 27.8B	202.0
14 / 56	MiniMax M2 · 228.7B	160.6
15 / 60	Qwen3 235B A22B Thinking 2507 · 235.1B	-11.3
16 / 61	GPT OSS 120B · 116.8B	-21.5
17 / 62	MiniMax M2.5 · 228.7B	-23.2

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.

Vending-Bench 2: frequently asked questions

What is the best open LLM on Vending-Bench 2?: GLM 5.2 is the top open model on Vending-Bench 2, scoring 8313.8. Among all models tested — including proprietary ones — it ranks #7. The top model overall is GPT 6 Astra (unspecified) (OpenAI) at 15514.7.
What's the best Vending-Bench 2 model you can run on a 24 GB GPU?: Qwen3.5 27B is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 15 GB), scoring 202.0 on Vending-Bench 2.
Can open models match proprietary models on Vending-Bench 2?: Not quite on Vending-Bench 2: the strongest proprietary model (GPT 6 Astra (unspecified)) scores 15514.7, ahead of the best open model (GLM 5.2) at 8313.8 — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.