Reasoning
Blueprint-Bench 2 Leaderboard
Blueprint-Bench 2, from Andon Labs, asks agents to reconstruct apartment floor plans from interior photographs, with a persistent notepad across 50 apartments. The connectivity similarity score is normalized so that random performance is 0 and a perfect reconstruction is 1.
Source: epoch2 open models ranked+35 proprietaryData through Nov 2026
All models ranked on Blueprint-Bench 2
Proprietary / closed models are shown dimmed — you can't run them locally, but they show where the open field stands.
| # | Model | Score | VRAM @ Q4 |
|---|---|---|---|
| 1 | Gemini 4 Carbon (unspecified) · proprietary | 58.6% | API only |
| 2 | Gemini 4 Argon (unspecified) · proprietary | 54.4% | API only |
| 3 | Claude Opus 5.5 (unspecified) · proprietary | 51.2% | API only |
| 4 | GPT 6 Astra (unspecified) · proprietary | 49.7% | API only |
| 5 | GPT 6.1 Sol (unspecified) · proprietary | 43.2% | API only |
| 6 | GPT 6.1 Terra (unspecified) · proprietary | 42.1% | API only |
| 7 | Claude Fable 5.1 (unspecified) · proprietary | 41.9% | API only |
| 8 | Claude Sonnet 5.5 (unspecified) · proprietary | 39.3% | API only |
| 9 | Claude Fable 5 (unspecified) · proprietary | 38.6% | API only |
| 10 | Gemini 3.8 Flash (unspecified) · proprietary | 38.6% | API only |
| 11 | GPT 6 Sol (unspecified) · proprietary | 36.9% | API only |
| 12 | GPT 5.5 (unspecified) · proprietary | 36.2% | API only |
| 13 | Gemini 3.5 Flash (unspecified) · proprietary | 33.6% | API only |
| 14 | GPT 5.6 Sol (unspecified) · proprietary | 33.6% | API only |
| 15 | Grok 4.6 (unspecified) · proprietary | 33.2% | API only |
| 16 | Claude Haiku 5.5 (unspecified) · proprietary | 33.0% | API only |
| 17 | Grok 4.7 (unspecified) · proprietary | 32.5% | API only |
| 18 | Muse Wonder 1.0 (unspecified) · proprietary | 31.7% | API only |
| 19 | Gemini 3.6 Flash (unspecified) · proprietary | 31.2% | API only |
| 20 | GPT 6 Luna (unspecified) · proprietary | 31.2% | API only |
| 21 | GPT 5.6 Terra (unspecified) · proprietary | 30.8% | API only |
| 22 | Claude Opus 5 (unspecified) · proprietary | 30.4% | API only |
| 23 | Kimi K3 · 2779.9B | 29.5% | ~1674 GB |
| 24 | Grok 4.5 (unspecified) · proprietary | 27.3% | API only |
| 25 | GPT 5.4 (Mar 05, 2026, unspecified) · proprietary | 27.1% | API only |
| 26 | Gemini 3.1 Pro Preview · proprietary | 26.5% | API only |
| 27 | Claude Sonnet 5 (unspecified) · proprietary | 24.9% | API only |
| 28 | Claude Opus 4.7 (unspecified) · proprietary | 24.5% | API only |
| 29 | GPT 5.6 Luna (unspecified) · proprietary | 22.6% | API only |
| 30 | Claude Opus 4.8 (unspecified) · proprietary | 14.5% | API only |
| 31 | Claude Sonnet 4.6 (unspecified) · proprietary | 6.7% | API only |
| 32 | Kimi K2.6 · 1026.9B | 3.9% | ~620 GB |
| 33 | Claude Haiku 4.5 (Oct 01, 2025, unspecified) · proprietary | 0.0% | API only |
| 34 | Gemini 3 Flash Preview · proprietary | 0.0% | API only |
| 35 | Gemini Robotics Er 1.6 Preview · proprietary | 0.0% | API only |
| 36 | Grok 4.20 · proprietary | 0.0% | API only |
| 37 | Grok 4.3 (unspecified) · proprietary | 0.0% | API only |
Blueprint-Bench 2: frequently asked questions
- What is the best open LLM on Blueprint-Bench 2?
- Kimi K3 is the top open model on Blueprint-Bench 2, scoring 29.5%. Among all models tested — including proprietary ones — it ranks #23. The top model overall is Gemini 4 Carbon (unspecified) (Google DeepMind) at 58.6%.
- Can open models match proprietary models on Blueprint-Bench 2?
- Not quite on Blueprint-Bench 2: the strongest proprietary model (Gemini 4 Carbon (unspecified)) scores 58.6%, ahead of the best open model (Kimi K3) at 29.5% — but you can run the open one yourself.
Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.