NVIDIA · Hopper
Best AI Models for NVIDIA H200 NVL (141 GB)
Runs everything up to 120B MoE well.
141 GB HBM3e4800 GB/s836 TFLOPS FP16600 WHopper 2024
| # | Model | Speed | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 2 | Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 3 | Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 153 t/s | 153 tok/s | Q4_K_M · 21.7 GB | |
| 4 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 395 t/s | 395 tok/s | Q4_K_M · 16.5 GB | |
| 5 | Q4_K_M · 19.3 GB · 301 t/s | 301 tok/s | Q4_K_M · 19.3 GB |
| # | Model | Speed | Coding Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 2 | Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 153 t/s | 153 tok/s | Q4_K_M · 21.7 GB | |
| 3 | Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 320 t/s | 320 tok/s | Q4_K_M · 22.0 GB | |
| 4 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 395 t/s | 395 tok/s | Q4_K_M · 16.5 GB | |
| 5 | Q4_K_M · 19.3 GB · 301 t/s | 301 tok/s | Q4_K_M · 19.3 GB |
| # | Model | Speed | Reasoning Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 2 | Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 153 t/s | 153 tok/s | Q4_K_M · 21.7 GB | |
| 3 | Qwen3.5 27BAlibaba Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 4 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 395 t/s | 395 tok/s | Q4_K_M · 16.5 GB | |
| 5 | Q4_K_M · 19.3 GB · 301 t/s | 301 tok/s | Q4_K_M · 19.3 GB |
| # | Model | Speed | Math Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 2 | Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 3 | Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 320 t/s | 320 tok/s | Q4_K_M · 22.0 GB | |
| 4 | Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 153 t/s | 153 tok/s | overall | Q4_K_M · 21.7 GB |
| 5 | Q4_K_M · 19.3 GB · 301 t/s | 301 tok/s | Q4_K_M · 19.3 GB |
| # | Model | Speed ↓ | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 957 t/s | 957 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 491 t/s | 491 tok/s | Q4_K_M · 6.6 GB | |
| 3 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 465 t/s | 465 tok/s | Q4_K_M · 13.1 GB | |
| 4 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 395 t/s | 395 tok/s | Q4_K_M · 16.5 GB | |
| 5 | Q4_K_M · 19.3 GB · 301 t/s | 301 tok/s | Q4_K_M · 19.3 GB |
| # | Model | Speed | Vision Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 2 | Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 179 t/s | 179 tok/s | Q4_K_M · 17.9 GB | |
| 3 | Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 153 t/s | 153 tok/s | Q4_K_M · 21.7 GB | |
| 4 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 395 t/s | 395 tok/s | Q4_K_M · 16.5 GB | |
| 5 | Q4_K_M · 87.7 GB · 193 t/s | 193 tok/s | Q4_K_M · 87.7 GB |
See all 404 models on the H200 NVL →Best quant that fits · speeds estimated on this card
The H200 NVL runs everything up to 120B MoE well. Hopper generation, released 2024.
GenerationHopperreleased 2024
H200 NVL strengths and weaknesses
Strengths
- Runs 120B MoE models: GPT OSS 120B at 323 tok/s in 70.8 GB
- Room for long context: Llama 3.3 70B Instruct at Q4 uses 51.3 GB, leaving 89.7 GB free
- Reads long prompts fast: A 4K-token prompt on Qwen3.8 27B starts answering in 605 ms
Weaknesses
- 600 W under load: Plan a 1,200 W PSU; ≈$131 a year at 4 h a day
Output speed depends on memory bandwidth (4800 GB/s).
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB491tok/s
- 14BQwen3 14B · Q4_K_M · 11.9 GB328tok/s
- 27–31BQwen3.8 27B · Q4_K_M · 17.9 GB179tok/s
- 35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB320tok/s
- 70BLlama 3.3 70B Instruct · Q4_K_M · 51.3 GB67.0tok/s
- 120B MoEGPT OSS 120B · Q4_K_M · 70.8 GB323tok/s
Estimated speeds: Q4_K_M, 4K context, one user. How we estimate
All 1295 models on the H200 NVL
Featured onlyBest quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
Apertus 8B Instruct 2509ChatFunctions | Q4_K_M | 7.3 GB | 577.8 t/s | 66K | S100, Runs great | |
| Q4_K_M | 0.1 GB | 24000.0 t/s | 2K | S100, Runs great | ||
GPT Neo 125MChat | BF16 | 0.3 GB | 9454.5 t/s | 2K | S100, Runs great | |
Gemma 3n E2B ITVision | Q4_K_M | 5.9 GB | 869.1 t/s | — | S100, Runs great | |
GritLM 7BChat | Q4_K_M | 6.8 GB | 635.4 t/s | 33K | S100, Runs great | |
Bloom 560MChat | Q4_K_M | 1.6 GB | 8432.4 t/s | — | S100, Runs great | |
| Q4_K_M | 0.6 GB | 7255.8 t/s | 8K | S100, Runs great | ||
| Q4_K_M | 1.9 GB | 1695.7 t/s | 8K | S100, Runs great | ||
GPT OSS Safeguard 20BChatFunctions | Q4_K_M | 13.5 GB | 440.9 t/s | 131K | S100, Runs great | |
| Q4_K_M | 6.2 GB | 578.8 t/s | 8K | S100, Runs great | ||
| Q4_K_M | 6.2 GB | 578.8 t/s | 8K | S100, Runs great | ||
Vicuna 13B V1.3Chat | Q4_K_M | 8.6 GB | 363.6 t/s | 2K | S100, Runs great | |
Mellum2 12B A2.5B ThinkingChatFunctions | Q4_K_M | 7.8 GB | 491.9 t/s | 131K | S100, Runs great | |
| Q4_K_M | 17.3 GB | 184.8 t/s | 262K | S99, Runs great | ||
Qwen2.5 32B InstructChatFunctions | Q4_K_M | 24.3 GB | 152.2 t/s | 33K | S97, Runs great |
Rent NVIDIA H200 NVL in the cloud
Hourly rate on demand for one H200 NVL.
Vast.aiMarketplace
from$4.89/hr
Cost per million tokens
Output-token cost on an H200 NVL rented at $4.89/hr, one request at a time. Batch several users on the same card and the cost per token drops several-fold.
| Model | Quant | Speed | Per 1M output |
|---|---|---|---|
| NVIDIA Nemotron 3.5 Lightning 30B A3B BF16 | 301 tok/s | $4.51$0.027 input | |
| Qwen3.8 27B | 179 tok/s | $7.59$0.20 input | |
| Muse Glimmer 30B | 170 tok/s | $7.99$0.22 input | |
| Gemma 4 31B IT | 153 tok/s | $8.88$0.23 input | |
| Qwen3.8 Flash Next | 69 tok/s | $19.72$0.44 input |
Specs
- Architecture
- Hopper
- VRAM
- 141 GB HBM3e
- Memory bandwidth
- 4800.0 GB/s
- FP16 (dense)
- 835.50 TFLOPS
- CUDA cores
- 16,896
- Tensor cores
- 528
- Compute capability
- 9.0
- TDP
- 600 W
- Released
- 2024-11-18
H200 NVL FAQ
What is the largest model the H200 NVL can run?
Qwen3 235B A22B Thinking 2507 (235.1B) is the largest at 4-bit or better: at Q4_K_S it uses 134.1 GB of 141 GB, and runs at about 109 tok/s. At 3-bit, a bigger model fits too: DeepSeek V4 Flash (290.9B) at IQ3_M, 131.4 GB.
Can the H200 NVL run Qwen3.8 27B?
Yes. Qwen3.8 27B fits at Q4_K_M, using 17.9 GB of 141 GB, and it is the best model that runs well on this card.
How many tokens per second does the H200 NVL give?
About 179 tok/s on Qwen3.8 27B at Q4_K_M, with the first token after 0.6 s on a 4K-token prompt.
What power supply does the H200 NVL need?
Plan a 1200 W PSU (2× the 600 W TDP).
Is 141 GB enough for coding and agents?
Yes for coding: Qwen3.8 27B, the best of the 14 coding models that fit, runs at about 179 tok/s.