What is the best open LLM on Blueprint-Bench 2?

Kimi K3 is the top open model on Blueprint-Bench 2, scoring 29.5%. Among all models tested — including proprietary ones — it ranks #23. The top model overall is Gemini 4 Carbon (unspecified) (Google DeepMind) at 58.6%.

Can open models match proprietary models on Blueprint-Bench 2?

Not quite on Blueprint-Bench 2: the strongest proprietary model (Gemini 4 Carbon (unspecified)) scores 58.6%, ahead of the best open model (Kimi K3) at 29.5% — but you can run the open one yourself.

Reasoning

Blueprint-Bench 2 Leaderboard

Name: Blueprint-Bench 2 — open LLM scores
Creator: epoch

Blueprint-Bench 2, from Andon Labs, asks agents to reconstruct apartment floor plans from interior photographs, with a persistent notepad across 50 apartments. The connectivity similarity score is normalized so that random performance is 0 and a perfect reconstruction is 1.

Source: epoch2 open models ranked+35 proprietaryData through Nov 2026

Open models All models

Open models ranked on Blueprint-Bench 2

# shows rank among open models / rank overall (including proprietary).

#	Model	Score	VRAM @ Q4
1 / 23	Kimi K3 · 2779.9B	29.5%	~1674 GB
2 / 32	Kimi K2.6 · 1026.9B	3.9%	~620 GB

Blueprint-Bench 2: frequently asked questions

What is the best open LLM on Blueprint-Bench 2?: Kimi K3 is the top open model on Blueprint-Bench 2, scoring 29.5%. Among all models tested — including proprietary ones — it ranks #23. The top model overall is Gemini 4 Carbon (unspecified) (Google DeepMind) at 58.6%.
Can open models match proprietary models on Blueprint-Bench 2?: Not quite on Blueprint-Bench 2: the strongest proprietary model (Gemini 4 Carbon (unspecified)) scores 58.6%, ahead of the best open model (Kimi K3) at 29.5% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.