NVIDIA · 72x B200 + 36x Grace · Server
Best AI Models for NVIDIA GB200 NVL72 (13.4 TB)
Runs models up to 2.8T parameters, like Kimi K3.1 Pro.
13.4 TB HBM576 TB/s180 PFLOPS FP16120 kW2025
| # | Model | Speed | llmrun Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Kimi K3.1 ProMoonshot AIHow to run → Q4_K_M · 1712.1 GB · 170 t/s | 170 tok/s | Q4_K_M · 1712.1 GB | |
| 2 | Inkling 2 SmallThinking Machines Q4_K_M · 359.1 GB · 276 t/s | 276 tok/s | Q4_K_M · 359.1 GB | |
| 3 | DeepSeek V4.1 Flash 1017DeepSeek Q4_K_M · 458.4 GB · 376 t/s | 376 tok/s | Q4_K_M · 458.4 GB | |
| 4 | MiMo V2.6 Pro RLXiaomi Q4_K_M · 614.1 GB · 198 t/s | 198 tok/s | Q4_K_M · 614.1 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 147 t/s | 147 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed | Coding Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Kimi K3.1 ProMoonshot AIHow to run → Q4_K_M · 1712.1 GB · 170 t/s | 170 tok/s | Q4_K_M · 1712.1 GB | |
| 2 | DeepSeek V4.1 Flash 1017DeepSeek Q4_K_M · 458.4 GB · 376 t/s | 376 tok/s | Q4_K_M · 458.4 GB | |
| 3 | Kimi K3Moonshot AI Q4_K_M · 1670.0 GB · 170 t/s | 170 tok/s | Q4_K_M · 1670.0 GB | |
| 4 | GLM 5.3Z.ai Q4_K_M · 453.8 GB · 182 t/s | 182 tok/s | Q4_K_M · 453.8 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 147 t/s | 147 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed | Reasoning Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Q4_K_M · 458.4 GB · 376 t/s | 376 tok/s | Q4_K_M · 458.4 GB | |
| 2 | Inkling 2 SmallThinking Machines Q4_K_M · 359.1 GB · 276 t/s | 276 tok/s | Q4_K_M · 359.1 GB | |
| 3 | Kimi K3.1 ProMoonshot AI Q4_K_M · 1712.1 GB · 170 t/s | 170 tok/s | Q4_K_M · 1712.1 GB | |
| 4 | DeepSeek V4.1 FlashDeepSeek Q4_K_M · 458.4 GB · 376 t/s | 376 tok/s | Q4_K_M · 458.4 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 147 t/s | 147 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed | Math Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Kimi K3Moonshot AIHow to run → Q4_K_M · 1670.0 GB · 170 t/s | 170 tok/s | Q4_K_M · 1670.0 GB | |
| 2 | DeepSeek V4.1 Flash 1017DeepSeek Q4_K_M · 458.4 GB · 376 t/s | 376 tok/s | Q4_K_M · 458.4 GB | |
| 3 | DeepSeek V4.1 FlashDeepSeek Q4_K_M · 458.4 GB · 376 t/s | 376 tok/s | Q4_K_M · 458.4 GB | |
| 4 | GLM 5.3Z.ai Q4_K_M · 453.8 GB · 182 t/s | 182 tok/s | Q4_K_M · 453.8 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 147 t/s | 147 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed ↓ | llmrun Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 1595 t/s | 1595 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 818 t/s | 818 tok/s | Q4_K_M · 6.6 GB | |
| 3 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 550 t/s | 550 tok/s | Q4_K_M · 13.1 GB | |
| 4 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 459 t/s | 459 tok/s | Q4_K_M · 16.5 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 147 t/s | 147 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed | Agents Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Q4_K_M · 458.4 GB · 376 t/s | 376 tok/s | Q4_K_M · 458.4 GB | |
| 2 | Kimi K3.1 ProMoonshot AI Q4_K_M · 1712.1 GB · 170 t/s | 170 tok/s | Q4_K_M · 1712.1 GB | |
| 3 | GLM 5.3Z.ai Q4_K_M · 453.8 GB · 182 t/s | 182 tok/s | Q4_K_M · 453.8 GB | |
| 4 | MiMo V2.6 Pro RLXiaomi Q4_K_M · 614.1 GB · 198 t/s | 198 tok/s | Q4_K_M · 614.1 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 147 t/s | 147 tok/s | Q4_K_M · 182.3 GB |
See all 455 models on the NVIDIA GB200 NVL72 →Bars = traction for models without a Score yet
The NVIDIA GB200 NVL72 runs models up to 2.8T parameters, like Kimi K3.1 Pro. A 4K-token prompt waits 80 ms before the first word on Kimi K3.1 Pro. 72x B200 + 36x Grace (Blackwell), released 2025.
For models13.4 TBall of it
Chip72x B200 + 36x Gracereleased 2025
NVIDIA GB200 NVL72 strengths and weaknesses
Strengths
- Runs 120B MoE models: GPT OSS 120B at 378 tok/s in 70.8 GB
- Room for long context: Llama 3.3 70B Instruct at Q4 uses 51.3 GB, leaving 13.3 TB free
- Reads long prompts fast: A 32K-token prompt on Kimi K3.1 Pro starts answering in 660 ms
- FP8 and NVFP4 run natively: Blackwell takes FP8 and NVFP4 checkpoints in vLLM as published
Memory for models
13.4 TB of 13.4 TB120B MoE70.8 GB2780B1712.1 GB
For models 13.4 TBModel size at Q4
Everything on the size ladder fits. Kimi K3.1 Pro and Inkling 2 Small together use 2.1 TB and leave 11.3 TB free.
The same chip from other makers first, then machines and GPUs people weigh against it, each with the one reason it is worth a look. Speeds on Qwen3.6 35B A3B.
Step down to 508.5 GBMac Studio M5 Ultra (512 GB)Apple · 508.5 GB for models · M5 Ultra
▼ 76% speed▼ 12.9 TBabout $22,299
Step down to 508.5 GBMac Studio M3 Ultra (512 GB)Apple · 508.5 GB for models · M3 Ultra▼ 79% speed▼ 12.9 TBabout $9,499
Step down to 188.5 GBMac Studio M2 Ultra (192 GB)Apple · 188.5 GB for models · M2 Ultra▼ 79% speed▼ 13.2 TBabout $5,599
Step down to 124.5 GBMac Studio M4 Max (128 GB)Apple · 124.5 GB for models · M4 Max▼ 82% speed▼ 13.3 TBabout $3,499
Step down to 24 GBRTX 4090NVIDIA · 24 GB VRAM · GPU▼ 58% speed▼ 13.4 TBabout $1,599rent $0.34/hr
Step down to 32 GBRTX 5090NVIDIA · 32 GB VRAM · GPU▼ 41% speed▼ 13.4 TBabout $5,502rent $0.50/hr
Output speed depends on memory bandwidth: each model runs on as few of the 72 GPUs as hold it, at 8 TB/s per GPU. Grey rows need more than the 13.4 TB available.
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB818tok/s
- 14BQwen3 14B · Q4_K_M · 11.9 GB547tok/s
- 27–31BQwen3.8 27B · Q4_K_M · 17.9 GB299tok/s
- 35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB363tok/s
- 70BLlama 3.3 70B Instruct · Q4_K_M · 51.3 GB112tok/s
- 120B MoEGPT OSS 120B · Q4_K_M · 70.8 GB378tok/s
Estimated speeds: Q4_K_M, 4K context, one user. How we estimate
All 1392 models on the NVIDIA GB200 NVL72
Featured onlyBest quant that fits in 13392 GB, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
| Q4_K_M | 7.3 GB | 963.0 t/s | 66K | S100, Runs great | ||
Qwen3.5 4B GLB 3D CoderChatCodeFunctions | Q4_K_M | 3.4 GB | 1630.1 t/s | 262K | S100, Runs great | |
| BF16 | 5.0 GB | 1050.5 t/s | 262K | S100, Runs great | ||
Kimi K2.6 Eagle3Chat | Q4_K_M | 1.9 GB | 3561.6 t/s | 262K | S100, Runs great | |
LFM2.5 8B A1B Opus DistilChatReasoning | Q4_K_M | 5.6 GB | 653.7 t/s | 128K | S100, Runs great | |
GPT X2 125MChat | BF16 | 0.6 GB | 8125.0 t/s | 1K | S100, Runs great | |
MiniCPM OCR 1.0VisionFunctions | Q4_K_M | 1.5 GB | 4601.8 t/s | 262K | S100, Runs great | |
| BF16 | 34.8 GB | 465.4 t/s | 4K | S100, Runs great | ||
Granite 4.0 H SmallChatFunctions | Q4_K_M | 22.3 GB | 307.8 t/s | 131K | S100, Runs great | |
Qwen3.5 4B Claude Opus 4.6 Distilled HereticChatReasoning | BF16 | 9.7 GB | 544.5 t/s | 262K | S100, Runs great | |
LocoTrainer 4BChatCodeFunctions | Q4_K_M | 4.2 GB | 1793.1 t/s | 262K | S100, Runs great | |
HebatronChat | Q4_K_M | 19.3 GB | 304.8 t/s | 262K | S100, Runs great | |
Llama 3.1 Nemotron 70B Instruct HFChatFunctions | Q4_K_M | 48 GB | 120.1 t/s | 131K | S95, Runs great | |
| Q4_K_M | 55.8 GB | 93.5 t/s | 1049K | S90, Runs great | ||
Ouro 1.4BChatReasoning | BF16 | 6.4 GB | 1456.6 t/s | 66K | S100, Runs great |
Specs
- Chip
- 72x B200 + 36x Grace · NVIDIA B200 (GB200 Superchip)
- CPU cores
- 2592
- Memory
- 13.4 TB
- For models
- 13.4 TB
- Memory bandwidth
- 576 TB/s
- FP16 (dense)
- 180 PFLOPS
- Power
- 120 kW max
- Size
- rack (MGX, 18 compute trays + 9 NVLink switch trays
- Released
- 2025-02-04
NVIDIA GB200 NVL72 FAQ
How much memory is available for AI on the NVIDIA GB200 NVL72?
All 13.4 TB can hold model weights.
What is the largest model the NVIDIA GB200 NVL72 can run?
Kimi K3.1 Pro (2780.1B) is the largest at 4-bit or better: at Q4_K_M it uses 1712.1 GB, 13% of the available memory, and runs at about 170 tok/s.
How long does the NVIDIA GB200 NVL72 take to read a long prompt?
On Kimi K3.1 Pro, about 80 ms for a 4K-token prompt and 660 ms for a 32K-token document before the first word appears.