LLM Benchmarks
Public benchmark scores for open models you can actually run — every leaderboard mapped to the hardware it takes. Find the strongest model that fits your GPU.
New here? Read about benchmarks to learn what each one measures.
65 leaderboards5 skill areas1,984 ranked model scores
llmrun Composite Score
One open-ended quality index for open models, fitted across every public leaderboard we track. 100 is the average model of the reference field.
Best you can run
- #1Qwen3.8 27B17.9 GB at Q4_K_M
- #2Qwen3.6 27B17.9 GB at Q4_K_M
- #3Gemma 4 31B IT21.7 GB at Q4_K_M
VRAM at Q4_K_M with room for a 16K-token context; longer ones take extra memory.
Compiled from 45 benchmarks sourced from 4 providers; latest update on 17 Oct 2026
Reasoning
Multi-step logic and problem-solving — can the model think through a hard problem rather than recall an answer.
- GPQA DiamondIndustry standard300 models · 98 openKimi K3 93.1%open · best proprietary Gemini 4 Argon (high) 96.4%
- ARC-AGIIndustry standard232 models · 19 openDeepSeek V4.1 Flash 94.5%open · best proprietary Claude Fable 5 (xhigh) 98.5%
- ARC-AGI-2Industry standard222 models · 19 openDeepSeek V4.1 Flash 72.9%open · best proprietary GPT 6 Astra Max 95.0%
- LiveBench ReasoningIndustry standard70 models · 19 openKimi K3 90.7open · best proprietary GPT 6 Astra Max 92.7
- LiveBench 2026 ReasoningIndustry standard48 models · 15 openKimi K3 90.7open · best proprietary Gemini 4 Argon 95.1
- 50 models · 38 openDeepSeek v3 87.5%open · best proprietary Gemini 1.5 Pro 001 89.2%
- 101 models · 25 openDeepSeek V4 Flash 0731 61.1%open · best proprietary Claude Fable 5 Max 81.9%
- 219 models · 66 openMiMo V2.6 Pro RL 26.6%open · best proprietary GPT 5.6 Sol Max 32.3%
- 79 models · 51 openLlama 3.1 405B 89.2%open · best proprietary Claude 3 Opus (Feb 29, 2024) 88.5%
- 59 models · 40 openPhi 3.5 MoE Instruct 88.6%open · best proprietary GPT 4o Mini (Jul 18, 2024) 88.7%
- 54 models · 16 openMiMo V2.6 Pro RL 59.5%open · best proprietary Gemini 4 Argon (unspecified) 82.4%
- 146 models · 14 openDeepSeek V4.1 Flash 48.0%open · best proprietary GPT 6 Astra Promax 76.0%
- 133 models · 18 openDeepSeek V4.1 Flash 43.0%open · best proprietary GPT 6 Astra Max 84.0%
- 214 models · 68 openDeepSeek V4 Pro 0813 93.9%open · best proprietary Claude Opus 5.5 Max 98.9%
- 176 models · 54 openGLM 5.3 55.5%open · best proprietary Claude Opus 5.5 Max 68.2%
- 40 models · 15 openDeepSeek R1 34.9%open · best proprietary GPT 6 Astra Max 68.3%
- 58 models · 25 openKimi K2.5 86.1%open · best proprietary GPT 5 (Aug 07, 2025, medium) 97.2%
- 22 models · 7 openKimi K2.5 19.3%open · best proprietary GPT 5.4 (Mar 05, 2026, xhigh) 27.9%
- 17 models · 6 openDeepSeek V4 Pro 13.5%open · best proprietary GPT 5.5 (high) 22.2%
- 48 models · 15 openDeepSeek V4 Flash Vision Exp 79.5open · best proprietary Gemini 4 Argon 85.2
- 48 models · 15 openMiMo V2.6 Pro RL 88.0open · best proprietary Claude Fable 5.1 100.0
- 82 models · 26 openKimi K3 61.9open · best proprietary O3 (Apr 16, 2025, unspecified) 62.5
- 69 models · 17 openGLM 5.2 8313.8open · best proprietary GPT 6 Astra (unspecified) 15514.7
- 34 models · 3 openDeepSeek V4.1 Flash 34.2%open · best proprietary Claude Opus 5.5 Max 83.3%
- 27 models · 2 openGLM 5.2 9.5%open · best proprietary GPT 6 Astra Max 76.2%
What these 25 boards measure
- GPQA Diamond · via epoch
- GPQA Diamond is a set of extremely hard, graduate-level science questions (physics, chemistry, biology) written by domain experts and filtered so that skilled non-experts with web access still fail. It measures genuine reasoning rather than memorization.
- ARC-AGI · via epoch
- ARC-AGI tests fluid, abstract reasoning on small visual grid puzzles where each task follows a novel rule the model must infer from a few examples. It deliberately resists memorization and is one of the most-watched measures of general reasoning progress.
- ARC-AGI-2 · via epoch
- ARC-AGI-2 is the second-generation ARC Prize puzzle set, rebuilt so that tasks solved by brute-force search or memorization were removed and every problem still requires inferring a novel rule from a handful of examples. It is markedly harder than the original ARC-AGI, and most models — open and closed alike — score near zero.
- LiveBench Reasoning · via livebench
- LiveBench Reasoning measures logical, multi-step reasoning using contamination-free questions that are refreshed regularly, so models cannot have trained on the test set.
- LiveBench 2026 Reasoning · via livebench
- LiveBench 2026 Reasoning measures logical, multi-step problem solving. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
- BIG-Bench Hard · via epoch
- BIG-Bench Hard (BBH) is the 23-task subset of BIG-Bench that earlier models found hardest — multi-step reasoning, logic, and word problems. It's a long-standing standard for tracking reasoning ability across model generations.
- SimpleBench · via epoch
- SimpleBench is a set of everyday, common-sense and trick questions that humans answer easily but language models often get wrong. It probes basic reasoning and robustness rather than specialist knowledge.
- CritPt · via epoch
- CritPt (Critical Points) tests graduate-level physics problem-solving on research-style questions spanning mechanics, electromagnetism, quantum mechanics, and more. It was created by physicists to probe whether a model can reason through multi-step physics rather than pattern-match textbook answers.
- WinoGrande · via epoch
- WinoGrande is a large-scale commonsense-reasoning test built around pronoun resolution: the model must pick which of two nouns a pronoun refers to in a sentence, using real-world knowledge to disambiguate. It was designed by AI2 to resist the shortcuts that let earlier models game its predecessor, Winograd.
- PIQA · via epoch
- PIQA (Physical Interaction QA) asks a model to choose the more sensible of two ways to accomplish an everyday physical task, testing practical, physical-world commonsense that isn't well captured by purely textual reasoning tests.
- APEX-Agents · via epoch
- APEX-Agents (AI Productivity Index) is Mercor's evaluation of AI agents on long-horizon, multi-application professional-services work — investment-banking, management-consulting, and corporate-law style tasks — graded against expert-produced solutions. The score is Pass@1, the fraction solved correctly on the first attempt.
- Chess Puzzles · via epoch
- Chess Puzzles is an Epoch AI evaluation that asks models to find the winning move in tactical chess positions drawn from real games. Because each puzzle has one verifiable correct answer and no natural-language shortcut, it isolates concrete multi-step planning.
- Mystery Game Puzzles · via epoch
- Mystery Game Puzzles is an Epoch AI evaluation of multi-step deductive reasoning: the model works through a mystery or logic-game scenario to arrive at the correct solution. It probes sustained, structured inference rather than single-step question answering.
- DTBench · via epoch
- DTBench is a set of handcrafted, expert-validated multiple-choice questions on decision theory — Newcomb-like problems and related puzzles — built by the Conceptual Reasoning Index project with Anthropic. It tests whether a model can follow an abstract argument rather than recall a fact.
- LMCA · via epoch
- LMCA (Language Model Conceptual Argumentation) scores models on expert-rated arguments across philosophy, decision theory, and AI risk — domains where there is no empirical feedback loop to check an answer against. It is part of the Conceptual Reasoning Index, built with Anthropic by researchers Emery Cooper and Caspar Oesterheld.
- BALROG · via epoch
- BALROG (Benchmarking Agentic LLM and VLM Reasoning On Games) tests long-horizon agentic reasoning across six game environments — BabyAI, Crafter, TextWorld, Baba Is AI, MiniHack and NetHack — scoring the average progress a model makes toward completing each game. It was built by UCL's DARK lab and collaborators.
- Fiction.LiveBench · via epoch
- Fiction.LiveBench is a long-context comprehension test: models answer questions that require tracking characters, events and relationships across a long story. llmrun shows the 16k-token score, the column Epoch AI reports as its headline.
- CL-bench · via epoch
- CL-bench, built by Tencent Hunyuan and Fudan University's NLP group, tests whether a model can learn new domain knowledge, rule systems and procedures purely from a long provided context — not from pretraining — and apply them correctly, across 1,899 tasks drawn from 500 scenarios. An LLM judge grades each task against dozens of fine-grained rubrics, and a task counts as solved only when every rubric passes.
- CL-bench Life · via epoch
- CL-bench Life, a follow-up from the CL-bench authors, tests the same context-learning skill on messier real-life material — fragmented notes, casual conversations, and behavioral logs — across 405 human-curated context-task pairs. It uses the same rubric-based LLM-judge scoring as CL-bench, requiring every rubric on a task to pass for it to count as solved.
- LiveBench 2026 Data Analysis · via livebench
- LiveBench 2026 Data Analysis measures questions answered by working with tables and structured data. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
- LiveBench 2026 Browser · via livebench
- LiveBench 2026 Browser measures tasks a model completes by operating a web browser. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
- ForecastBench · via epoch
- ForecastBench, from the Forecasting Research Institute, asks models to predict real-world future events, with questions that only resolve after the model's training cutoff so the answers can't have been memorised. It is scored as a difficulty-adjusted Brier index, where higher is better.
- Vending-Bench 2 · via epoch
- Vending-Bench 2, from Andon Labs, is a long-horizon agent benchmark in which the model runs a simulated vending-machine business for a year, handling suppliers, pricing and cash flow. The score is the final money balance in US dollars.
- Furniture Assembly · via epoch
- Furniture Assembly, built by Epoch AI, tests spatial reasoning with 60 photos of partly built IKEA furniture and the assembly manual: the model must say whether the build is correct so far and, if not, name the mistaken step and describe the error. Scores are accuracy across the photos, and a human baseline averaged 63%.
- EBR-bench · via epoch
- EBR-bench, from Epoch AI, tests whether models learn from experience by having them play the board game Earthborne Rangers over repeated playthroughs, with the rulebook, card database and a persistent note-taking tool. Scores stay roughly flat across plays and well below an expert human baseline.
Coding
Writing, editing, and fixing real code — measured by whether the resulting program actually runs and passes tests.
- LiveBench CodingIndustry standard70 models · 19 openKimi K3 81.5open · best proprietary Claude Sonnet 5.5 Max 91.4
- LiveBench 2026 CodingIndustry standard48 models · 15 openQwen3.8 27B 86.8open · best proprietary Claude Fable 5 98.3
- LMArena WebDevIndustry standard138 models · 52 openKimi K3 1655.2open · best proprietary Claude Opus 5.5 (Max) 1807.3
- SWE-bench VerifiedIndustry standard162 models · 20 openMiniMax M2.5 75.8%open · best proprietary Sonar Foundation Agent + Claude 4.5 Opus 79.2%
- 48 models · 9 openKimi K2 Thinking 63.4%open · best proprietary Claude 4.5 Opus (high reasoning) 76.8%
- 57 models · 16 openGLM 5 52.4%open · best proprietary GPT 5.5 (unspecified) 84.7%
- 69 models · 18 openDeepSeek V3.2 Exp 74.2%open · best proprietary GPT 5 (Aug 07, 2025, high) 88.0%
- 13 models · 4 openGLM 5 69.7%open · best proprietary Gemini 3 Flash 72.7%
- 80 models · 3 openQwen3 Coder 30B A3B Instruct 49.7%open · best proprietary ExpeRepair-v1.0 + Claude 4 Sonnet 60.3%
- 207 models · 66 openMiMo V2.6 Pro RL 60.9%open · best proprietary Claude Opus 5.5 Max 66.9%
- 162 models · 46 openKimi K3 82.6%open · best proprietary GPT 6 Astra Promax 93.6%
- 46 models · 12 openKimi K3 44.2%open · best proprietary Claude Opus 5.5 (medium) 54.6%
- 68 models · 5 openGLM 5.3 69.0%open · best proprietary GPT 6 Astra (xhigh) 74.1%
- 42 models · 3 openKimi K2 Instruct 4.9%open · best proprietary Claude Fable 5.1 (unspecified) 88.2%
- 28 models · 13 openKimi K3 95.0%open · best proprietary Claude Fable 5 (high) 95.0%
- 21 models · 4 openGLM 5.3 30.2%open · best proprietary GPT 6 Astra Max 65.5%
- 48 models · 15 openMiMo V2.6 Pro RL 47.5open · best proprietary Claude Opus 5.5 85.0
- 68 models · 2 openGLM 5.3 42.6%open · best proprietary Claude Opus 5.5 Max 57.8%
- 122 models · 43 openKimi K3 1524.5open · best proprietary GPT 6 Astra Max 2951.3
- 9 models · 3 openKimi K2.6 18.4%open · best proprietary Claude Mythos Preview Early 73.8%
- 46 models · 9 openKimi K2 Thinking 59.2%open · best proprietary Claude Mythos Preview Early 85.2%
- 48 models · 15 openMiMo V2.6 Pro RL 78.5open · best proprietary Claude Opus 5.5 86.9
What these 22 boards measure
- LiveBench Coding · via livebench
- LiveBench Coding evaluates code generation and completion on fresh, contamination-free programming tasks that are updated regularly.
- LiveBench 2026 Coding · via livebench
- LiveBench 2026 Coding measures whether a model writes correct code for programming problems. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
- LMArena WebDev · via lmarena
- Code Arena (formerly WebDev Arena) is Arena's head-to-head coding leaderboard: two models each build a real web app or UI from the same prompt, and people vote blind on which result is better. The rating shown is a relative Bradley-Terry score, not a percentage — compare it only to other models on this board.
- SWE-bench Verified · via swebench
- SWE-bench Verified tests whether a model can resolve real GitHub issues from popular open-source Python projects, scored on the official swebench.com leaderboard as the percentage of human-validated issues actually fixed. It is the headline measure of practical, agentic software-engineering ability — where open-weight models like Qwen3-Coder, GLM-4.6, Kimi K2 and DeepSWE are now competitive with the frontier.
- SWE-bench Bash Only · via swebench
- SWE-bench Bash Only runs the SWE-bench Verified issues through a minimal, single-tool bash agent — no specialised scaffolding — so the score reflects the model's own agentic coding ability rather than the harness around it. A cleaner, harder read on raw software-engineering skill.
- Terminal-Bench · via epoch
- Terminal-Bench measures whether a model can complete real, end-to-end tasks in a command-line environment — running commands, editing files, and chaining steps — making it an agentic test of practical software skill.
- Aider Polyglot · via epoch
- The Aider Polyglot benchmark measures real-world coding across several programming languages: the model edits code to solve Exercism exercises, and is scored on whether the final solution actually runs and passes the tests.
- SWE-bench Multilingual · via swebench
- SWE-bench Multilingual extends SWE-bench beyond Python to real GitHub issues across many programming languages, measuring whether a model can fix bugs in codebases written in Java, Go, Rust, TypeScript and more.
- SWE-bench Lite · via swebench
- SWE-bench Lite is a smaller, lower-cost subset of SWE-bench focused on self-contained bug fixes. It is the quickest of the SWE-bench boards to run and a common entry point for comparing coding agents.
- SciCode · via epoch
- SciCode asks a model to write real scientific-computing code — implementing multi-step numerical methods drawn from physics, chemistry, biology, and math research — and checks the output against reference solutions. It was built by a team of scientists and ML researchers to test coding ability on genuine research problems, not typical software tasks.
- WeirdML · via epoch
- WeirdML asks models to write machine-learning code for small, deliberately unusual datasets, then runs the code and scores the accuracy it actually achieves. Built by Håvard Tveit Ihle, it rewards practical problem-solving over recognizing a familiar task.
- FrontierCode · via epoch
- FrontierCode, from Cognition (the team behind Devin) with maintainers of open-source projects, gives an agent a real repository and issue and grades its patch on whether it's mergeable — held-out tests plus a maintainer-written rubric covering correctness, regression safety, style and project conventions. The Main set is the 100 hardest tasks.
- DeepSWE · via epoch
- DeepSWE, built by Datacurve, evaluates coding agents on 113 original software-engineering tasks across 91 repositories and five languages — written from scratch rather than adapted from real commits, so a model can't have seen the solution during pretraining, and graded Pass@1 with hand-written behavioral verifiers. It's a benchmark name only, unrelated to the open-weight DeepSWE-Preview coding model.
- GSO-Bench · via epoch
- GSO-Bench, built by a team of UC Berkeley-affiliated researchers, gives a coding agent a real codebase and a performance test and asks it to optimize runtime efficiency to match an expert developer's speedup without breaking correctness, across 102 tasks in 10 codebases and five languages. The scored metric, Opt@1, is the share of tasks where a single attempt reaches at least 95% of the human speedup while still passing all tests.
- Surface Evolver Bench · via epoch
- Surface Evolver Bench tests whether a model can write a correct simulation file for Surface Evolver, a physics tool that models surfaces shaped by tension, gravity and volume constraints, defining a 3D structure in its domain-specific format from only a few examples. Scores compare the resulting geometry — volume, area and energy — against reference solutions; the benchmark was built by an independent developer (GitHub: yhenon).
- FrontierSWE · via epoch
- FrontierSWE, from Proximal, tests coding agents on long-horizon software-engineering tasks spanning implementation, performance optimisation and open-ended research, each run for hours through an agent harness. The score is the average normalised task score across repeated runs of every task.
- LiveBench 2026 Long-Horizon · via livebench
- LiveBench 2026 Long-Horizon measures multi-step tasks that a model has to carry through many steps to finish. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
- CursorBench · via epoch
- CursorBench, from Cursor (Anysphere), measures agentic coding on tasks drawn from real Cursor engineering sessions rather than synthetic puzzles. Each model is run at several reasoning-effort levels, and Epoch AI reports the correctness score.
- ALE-Bench · via epoch
- ALE-Bench, from Sakana AI and AtCoder, tests long-horizon algorithmic optimisation using problems drawn from AtCoder Heuristic Contests, where there is no single correct answer and solutions are scored on quality. Results are reported as an AtCoder-style performance rating, where higher is better.
- ExploitBench · via epoch
- ExploitBench, from Carnegie Mellon University researchers Seunghyun Lee and David Brumley, scores AI agents on writing working exploits for 41 real, already-patched bugs in Chrome's V8 JavaScript engine, with modern mitigations switched on. Instead of pass/fail it grades 16 stages from crash up to arbitrary code execution, and Epoch AI reports the mean capability reached.
- METR Time Horizons · via epoch
- METR Time Horizons, from the AI-evaluation nonprofit METR, runs models on a suite of software-engineering and research tasks whose length is measured by how long they take skilled humans, from seconds to many hours. METR summarises each model as a time horizon: the human task length at which the model succeeds half the time. llmrun shows METR's average task-success score (0–1) as published by Epoch AI; higher means the model completes more of the suite, including longer tasks.
- LiveBench 2026 Design · via livebench
- LiveBench 2026 Design measures layout-design tasks (LiveBench's design_layout task). It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
Math
Symbolic and quantitative problem-solving, from competition math to multi-step calculation.
- AIME 2024/2025Industry standard283 models · 86 openDeepSeek V4 Pro 0813 98.6%open · best proprietary Gemini 4 Argon (high) 100.0%
- MATH Level 5Industry standard107 models · 39 openDeepSeek R1 0528 96.6%open · best proprietary GPT 5 (Aug 07, 2025, high) 98.1%
- LiveBench MathIndustry standard70 models · 19 openDeepSeek V4 Pro 0813 95.1open · best proprietary Claude Opus 5.5 Max 97.1
- FrontierMathIndustry standard114 models · 17 openKimi K3 72.2%open · best proprietary GPT 6.1 Sol Max 93.7%
- 76 models · 13 openKimi K3 39.0%open · best proprietary GPT 6.1 Sol Max 100.0%
- 93 models · 62 openDeepSeek Coder v2 Instruct 94.5%open · best proprietary GPT 4 (Mar 14) 92.0%
- 82 models · 26 openKimi K3 87.0%open · best proprietary Claude Fable 5.1 Max 100.0%
What these 7 boards measure
- AIME 2024/2025 · via epoch
- AIME (American Invitational Mathematics Examination) is a prestigious high-school competition of hard, integer-answer problems. It's a widely-cited yardstick for multi-step mathematical reasoning, here on the 2024–2025 papers.
- MATH Level 5 · via epoch
- MATH Level 5 covers the hardest tier of competition-mathematics problems, testing multi-step symbolic and quantitative reasoning.
- LiveBench Math · via livebench
- LiveBench Math measures mathematical problem-solving on contamination-free, regularly-refreshed questions, including competition-style problems.
- FrontierMath · via epoch
- FrontierMath is a benchmark of exceptionally hard, original research-level mathematics problems created with professional mathematicians. Even the strongest models solve only a small fraction, making it a frontier measure of genuine mathematical ability.
- FrontierMath Tier 4 · via epoch
- FrontierMath Tier 4 is the hardest tier of Epoch AI's research-level mathematics benchmark, holding problems that working mathematicians consider genuinely difficult research questions. Scores here stay low even for frontier models, making it the sharpest available read on mathematical ability.
- GSM8K · via epoch
- GSM8K is a classic set of grade-school math word problems requiring several arithmetic reasoning steps. Long a default sanity check for reasoning, top models now score near the ceiling, so it best separates smaller models.
- ProofBench · via epoch
- ProofBench evaluates a model's ability to write and check rigorous mathematical proofs, rather than just produce a final numeric answer. It probes step-by-step logical correctness, which is a harder and more distinct skill than solving for a number.
Knowledge
Breadth and depth of factual knowledge across many subjects, paired with the reasoning to apply it.
- MMLUIndustry standard136 models · 85 openDeepSeek v3 87.2%open · best proprietary GPT 4o (Nov 20, 2024) 88.1%
- Humanity's Last ExamIndustry standard51 models · 4 openKimi K2.5 24.4%open · best proprietary GPT 6 Astra (unspecified) 54.8%
- 91 models · 17 openMuse Spark Open 816B A42B 56.6%open · best proprietary Gemini 4 Argon (high) 76.0%
- 48 models · 15 openDeepSeek V4.1 Flash 85.3open · best proprietary Claude Opus 5 88.9
- 75 models · 47 openLlama 3.1 405B 89.2%open · best proprietary GPT 4 32K (Mar 14) 95.3%
- 257 models · 145 openMiniMax M2.1 88.0%open · best proprietary Gemini-3.1-Pro 91.2%
- 76 models · 56 openDeepSeek v3 95.3%open · best proprietary GPT 3.5 Turbo (Nov 06) 87.4%
- 76 models · 38 openStableBeluga2 89.4%open · best proprietary T5 11B 91.2%
- 41 models · 26 openPhi 3 Mini 4k Instruct 88.0%open · best proprietary GPT 3.5 Turbo (Nov 06) 86.0%
- 39 models · 19 openLlama 2 70B HF 87.6%open · best proprietary Claude 2.0 87.5%
What these 10 boards measure
- MMLU · via epoch
- MMLU (Massive Multitask Language Understanding) spans 57 subjects from history to law to medicine as multiple-choice questions. It is the long-standing default for broad knowledge and remains the most widely-reported general benchmark.
- Humanity's Last Exam · via epoch
- Humanity's Last Exam (HLE) is a set of extremely difficult, expert-written questions across many fields, designed so that even frontier models score low. It is built to stay hard as models improve, measuring the true knowledge frontier.
- SimpleQA · via epoch
- SimpleQA measures factual accuracy on short, fact-seeking questions with a single correct answer — directly probing how often a model is right versus confidently wrong (hallucination) on simple facts.
- LiveBench 2026 Research · via livebench
- LiveBench 2026 Research measures finding and combining information to answer research questions. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
- HellaSwag · via epoch
- HellaSwag is a commonsense benchmark that asks a model to pick the most plausible continuation of an everyday situation. The wrong options are adversarially chosen to fool models, testing grounded commonsense understanding.
- MMLU-Pro · via tigerlab
- MMLU-Pro is a harder, cleaned-up successor to MMLU with ten answer choices and more reasoning-heavy questions across 14 subjects, measuring broad knowledge and reasoning together.
- ARC Challenge · via epoch
- ARC Challenge is a set of grade-school science questions, curated by the Allen Institute for AI (AI2) to exclude ones that simple word-matching or retrieval can answer. It measures science knowledge paired with the reasoning needed to apply it.
- BoolQ · via epoch
- BoolQ is a reading-comprehension benchmark of naturally occurring yes/no questions, each paired with a short passage that contains the answer. It tests whether a model can extract and reason over information from real text rather than just recall facts.
- OpenBookQA · via epoch
- OpenBookQA is a science-question benchmark modeled on open-book exams: the model is given a small set of core science facts and must combine them with broader common knowledge to answer multi-step questions. It targets reasoning over recall.
- TriviaQA · via epoch
- TriviaQA is a large set of trivia questions paired with evidence documents, testing a model's factual recall and reading comprehension across a wide range of everyday topics.
Instruction Following
How precisely a model obeys explicit, verifiable instructions — format, length, and content constraints — and, more broadly, how much people prefer its responses head-to-head in blind human voting.
- LMArena TextIndustry standard405 models · 187 openKimi K3 1487.1open · best proprietary Gemini 4 Argon (High) 1523.9
What this board measures
- LMArena Text · via lmarena
- Arena (arena.ai, formerly LMArena / Chatbot Arena) ranks models by millions of blind, head-to-head human votes on real prompts. The Arena Score shown here is a Bradley-Terry rating computed with Arena's default style control, so higher means people preferred that model's answers more often — it's a relative rating, not a percentage, so compare it only to other models on this board.
Scores are aggregated from public benchmark sources and attributed on each leaderboard. llmrun does not run these benchmarks.