Smaug Flash — Benchmarks

Benchmark scores for Smaug Flash aggregated from public leaderboards (10 boards), with how it ranks among open models and how it compares with its nearest neighbours. See hardware requirements for what you need to run it.

No llmrun Composite Score for Smaug Flash: the score is only published for a model measured on at least 4 of the boards in the score panel, spanning at least 2 benchmark categories and including a reasoning or coding board — or on 3 boards that narrow its score to a 90% range of at most 15.6 points. Either way the results must come from at least two different benchmarks. Its individual scores below stand on their own — see how the llmrun Composite Score gets calculated.

Smaug Flash reasoning benchmarks

Multi-step logic and problem-solving — can the model think through a hard problem rather than recall an answer.

LiveBench Reasoning
LiveBench Reasoning measures logical, multi-step reasoning using contamination-free questions that are refreshed regularly, so models cannot have trained on the test set.
86.2
86.2
median 82.7open leader 90.7
Overall best: GPT 6 Astra Max 92.7
78th percentile · #5 of 19 open · #34 of 70 all
#5 of 19
#34 of 70
Independent
LiveBench · Jun 2026
LiveBench 2026 Reasoning
LiveBench 2026 Reasoning measures logical, multi-step problem solving. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
86.2
86.2
median 78.8open leader 90.7
Overall best: Gemini 4 Argon 95.1
86th percentile · #3 of 15 open · #27 of 48 all
#3 of 15
#27 of 48
Independent
LiveBench · Sep 2026

Other results (2)· leaderboard-only, not part of the llmrun Score

LiveBench 2026 Browser
LiveBench 2026 Browser measures tasks a model completes by operating a web browser. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
0.0
0.0
median 16.0open leader 88.0
Overall best: Claude Fable 5.1 100.0
11th percentile · #12 of 15 open · #44 of 48 all
#12 of 15
#44 of 48
Independent
LiveBench · Sep 2026
LiveBench 2026 Data Analysis
LiveBench 2026 Data Analysis measures questions answered by working with tables and structured data. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
79.1
79.1
median 72.8open leader 79.5
Overall best: Gemini 4 Argon 85.2
86th percentile · #3 of 15 open · #15 of 48 all
#3 of 15
#15 of 48
Independent
LiveBench · Sep 2026

Smaug Flash coding benchmarks

Writing, editing, and fixing real code — measured by whether the resulting program actually runs and passes tests.

LiveBench Coding
LiveBench Coding evaluates code generation and completion on fresh, contamination-free programming tasks that are updated regularly.
77.1
77.1
median 75.0open leader 81.5
Overall best: Claude Sonnet 5.5 Max 91.4
61st percentile · #8 of 19 open · #44 of 70 all
#8 of 19
#44 of 70
Independent
LiveBench · Jun 2026
LiveBench 2026 Coding
LiveBench 2026 Coding measures whether a model writes correct code for programming problems. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
60.1
60.1
median 65.5open leader 86.8
Overall best: Claude Fable 5 98.3
36th percentile · #10 of 15 open · #41 of 48 all
#10 of 15
#41 of 48
Independent
LiveBench · Sep 2026
LiveBench 2026 Long-Horizon
LiveBench 2026 Long-Horizon measures multi-step tasks that a model has to carry through many steps to finish. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
27.1
27.1
median 27.1open leader 47.5
Overall best: Claude Opus 5.5 85.0
50th percentile · #8 of 15 open · #35 of 48 all
#8 of 15
#35 of 48
Independent
LiveBench · Sep 2026

Other results (1)· leaderboard-only, not part of the llmrun Score

LiveBench 2026 Design
LiveBench 2026 Design measures layout-design tasks (LiveBench's design_layout task). It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
76.2
76.2
median 70.7open leader 78.5
Overall best: Claude Opus 5.5 86.9
64th percentile · #6 of 15 open · #24 of 48 all
#6 of 15
#24 of 48
Independent
LiveBench · Sep 2026

Smaug Flash math benchmarks

Symbolic and quantitative problem-solving, from competition math to multi-step calculation.

LiveBench Math
LiveBench Math measures mathematical problem-solving on contamination-free, regularly-refreshed questions, including competition-style problems.
90.1
90.1
median 86.8open leader 95.1
Overall best: Claude Opus 5.5 Max 97.1
83rd percentile · #4 of 19 open · #37 of 70 all
#4 of 19
#37 of 70
Independent
LiveBench · Jun 2026

Smaug Flash knowledge benchmarks

Breadth and depth of factual knowledge across many subjects, paired with the reasoning to apply it.

LiveBench 2026 Research
LiveBench 2026 Research measures finding and combining information to answer research questions. It belongs to the 2026 LiveBench release (first snapshot September 2, 2026), a redesigned LiveBench in which each of seven categories is scored by a single task and questions are refreshed to limit training-set contamination. It is a different test from the original LiveBench boards, so the scores are not comparable.
74.1
74.1
median 70.1open leader 85.3
Overall best: Claude Opus 5 88.9
71st percentile · #5 of 15 open · #22 of 48 all
#5 of 15
#22 of 48
Independent
LiveBench · Sep 2026

Scores aggregated from public benchmark sources (LiveBench: 10 boards) — each linked from the benchmark pages. llmrun does not run these benchmarks. Percentiles and ranks are among open models in the llmrun catalogue; the median and leader marks refer to the same field.