llmrun
Knowledge

SimpleQA Leaderboard

SimpleQA measures factual accuracy on short, fact-seeking questions with a single correct answer — directly probing how often a model is right versus confidently wrong (hallucination) on simple facts.

Source: epoch15 open models ranked+63 proprietaryData through Sep 2026

Open modelsAll models

Open models ranked on SimpleQA

# shows rank among open models / rank overall (including proprietary).

#ModelScore
1 / 16DeepSeek V4 Pro 0813 · 1650.5B
52.9%
2 / 19Kimi K3 · 2779.9B
50.6%
3 / 28DeepSeek V4 Pro · 1598.8B
47.0%
4 / 36GLM 5.3 · 753.3B
41.0%
5 / 38Qwen3 235B A22B Thinking 2507 · 235.1B
40.4%
6 / 39Inkling · 952.4B
40.3%
7 / 41Kimi K2.7 Code · 1026.9B
36.5%
8 / 43Kimi K2.6 · 1026.9B
34.9%
9 / 45Kimi K2.5 · 1026.9B
34.3%
10 / 46GLM 5.2 · 753.3B
34.2%
11 / 47GLM 5.1 · 753.9B
34.0%
12 / 49DeepSeek V4 Flash 0731 · 304.2B
33.6%
13 / 55GLM 4.7 · 358.3B
32.2%
14 / 66Inkling Small · 266.0B
19.1%
15 / 76Gemma 4 31B IT · 31.3B
10.4%

Score vs model size

Which models give the most quality for their size — the ones worth running locally.

100B1Tmodel size (log scale) →52.9%10.4%Kimi K3 · 2.8T · 50.6%Inkling · 952B · 40.3%Kimi K2.7 Code · 1T · 36.5%Kimi K2.6 · 1T · 34.9%Kimi K2.5 · 1T · 34.3%GLM 5.2 · 753B · 34.2%GLM 5.1 · 754B · 34.0%DeepSeek V4 Flash 0731 · 304B · 33.6%GLM 4.7 · 358B · 32.2%Inkling Small · 266B · 19.1%Gemma 4 31B IT · 31B · 10.4%Gemma 4 31B ITQwen3 235B A22B Thinking 2507 · 235B · 40.4%Qwen3 235B A22B Think…GLM 5.3 · 753B · 41.0%GLM 5.3DeepSeek V4 Pro · 1.6T · 47.0%DeepSeek V4 ProDeepSeek V4 Pro 0813 · 1.7T · 52.9%DeepSeek V4 Pro 0813
Each dot is a model. Up = higher score, left = smaller (easier to run locally). The dashed line marks the efficiency frontier — the best score you can get at each size or smaller.
  • Gemma 4 31B IT, 31B, score 10.4% — on the efficiency frontier (best score at its size or smaller).
  • Qwen3 235B A22B Thinking 2507, 235B, score 40.4% — on the efficiency frontier (best score at its size or smaller).
  • GLM 5.3, 753B, score 41.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Pro, 1.6T, score 47.0% — on the efficiency frontier (best score at its size or smaller).
  • DeepSeek V4 Pro 0813, 1.7T, score 52.9% — on the efficiency frontier (best score at its size or smaller).

SimpleQA: frequently asked questions

What is the best open LLM on SimpleQA?
DeepSeek V4 Pro 0813 is the top open model on SimpleQA, scoring 52.9%. Among all models tested — including proprietary ones — it ranks #16. The top model overall is GPT 6 Astra Max (OpenAI) at 75.6%.
What's the best SimpleQA model you can run on a 24 GB GPU?
Gemma 4 31B IT is the highest-scoring open model that fits in 24 GB at 4-bit quantization (about 17 GB), scoring 10.4% on SimpleQA.
Can open models match proprietary models on SimpleQA?
Not quite on SimpleQA: the strongest proprietary model (GPT 6 Astra Max) scores 75.6%, ahead of the best open model (DeepSeek V4 Pro 0813) at 52.9% — but you can run the open one yourself.

Scores aggregated from epoch. llmrun does not run this benchmark — see the source for methodology, or the about benchmarks for what it measures.