MiniMax M2.5 — Benchmarks

ARC-AGI-2 is the second-generation ARC Prize puzzle set, rebuilt so that tasks solved by brute-force search or memorization were removed and every problem still requires inferring a novel rule from a handful of examples. It is markedly harder than the original ARC-AGI, and most models — open and closed alike — score near zero.

ARC-AGI tests fluid, abstract reasoning on small visual grid puzzles where each task follows a novel rule the model must infer from a few examples. It deliberately resists memorization and is one of the most-watched measures of general reasoning progress.

CL-bench, built by Tencent Hunyuan and Fudan University's NLP group, tests whether a model can learn new domain knowledge, rule systems and procedures purely from a long provided context — not from pretraining — and apply them correctly, across 1,899 tasks drawn from 500 scenarios. An LLM judge grades each task against dozens of fine-grained rubrics, and a task counts as solved only when every rubric passes.

CL-bench Life, a follow-up from the CL-bench authors, tests the same context-learning skill on messier real-life material — fragmented notes, casual conversations, and behavioral logs — across 405 human-curated context-task pairs. It uses the same rubric-based LLM-judge scoring as CL-bench, requiring every rubric on a task to pass for it to count as solved.

Vending-Bench 2, from Andon Labs, is a long-horizon agent benchmark in which the model runs a simulated vending-machine business for a year, handling suppliers, pricing and cash flow. The score is the final money balance in US dollars.

SWE-bench Verified tests whether a model can resolve real GitHub issues from popular open-source Python projects, scored on the official swebench.com leaderboard as the percentage of human-validated issues actually fixed. It is the headline measure of practical, agentic software-engineering ability — where open-weight models like Qwen3-Coder, GLM-4.6, Kimi K2 and DeepSWE are now competitive with the frontier.

Terminal-Bench measures whether a model can complete real, end-to-end tasks in a command-line environment — running commands, editing files, and chaining steps — making it an agentic test of practical software skill.

SWE-bench Multilingual extends SWE-bench beyond Python to real GitHub issues across many programming languages, measuring whether a model can fix bugs in codebases written in Java, Go, Rust, TypeScript and more.

ALE-Bench, from Sakana AI and AtCoder, tests long-horizon algorithmic optimisation using problems drawn from AtCoder Heuristic Contests, where there is no single correct answer and solutions are scored on quality. Results are reported as an AtCoder-style performance rating, where higher is better.

ProofBench evaluates a model's ability to write and check rigorous mathematical proofs, rather than just produce a final numeric answer. It probes step-by-step logical correctness, which is a harder and more distinct skill than solving for a number.

MMLU-Pro is a harder, cleaned-up successor to MMLU with ten answer choices and more reasoning-heavy questions across 14 subjects, measuring broad knowledge and reasoning together.

Arena (arena.ai, formerly LMArena / Chatbot Arena) ranks models by millions of blind, head-to-head human votes on real prompts. The Arena Score shown here is a Bradley-Terry rating computed with Arena's default style control, so higher means people preferred that model's answers more often — it's a relative rating, not a percentage, so compare it only to other models on this board.