AMD · CDNA 4
Best AI Models for AMD Instinct MI350X (288 GB)
Runs everything up to 120B MoE well.
288 GB HBM3e8000 GB/s2307 TFLOPS FP161000 WCDNA 4 2025
Datacenter CDNA4 accelerator — cloud/server only. Listed for comparison.
| # | Model | Speed | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 186.5 GB · 87.1 t/s | 87.1 tok/s | Q4_K_M · 186.5 GB | |
| 2 | DeepSeek V4 Flash 0731DeepSeek Q4_K_M · 183.0 GB · 83.2 t/s | 83.2 tok/s | Q4_K_M · 183.0 GB | |
| 3 | GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 88.4 t/s | 88.4 tok/s | Q4_K_M · 205.2 GB | |
| 4 | Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 276 t/s | 276 tok/s | Q4_K_M · 17.9 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 53.7 t/s | 53.7 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed | Coding Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 186.5 GB · 87.1 t/s | 87.1 tok/s | Q4_K_M · 186.5 GB | |
| 2 | DeepSeek V4 Flash 0731DeepSeek Q4_K_M · 183.0 GB · 83.2 t/s | 83.2 tok/s | Q4_K_M · 183.0 GB | |
| 3 | Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 276 t/s | 276 tok/s | Q4_K_M · 17.9 GB | |
| 4 | GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 88.4 t/s | 88.4 tok/s | Q4_K_M · 205.2 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 53.7 t/s | 53.7 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed | Reasoning Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 183.0 GB · 83.2 t/s | 83.2 tok/s | Q4_K_M · 183.0 GB | |
| 2 | GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 88.4 t/s | 88.4 tok/s | Q4_K_M · 205.2 GB | |
| 3 | Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 276 t/s | 276 tok/s | Q4_K_M · 17.9 GB | |
| 4 | Inkling SmallThinking Machines Q4_K_M · 162.7 GB · 116 t/s | 116 tok/s | Q4_K_M · 162.7 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 53.7 t/s | 53.7 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed ↓ | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 1472 t/s | 1472 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 755 t/s | 755 tok/s | Q4_K_M · 6.6 GB | |
| 3 | Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 235 t/s | 235 tok/s | Q4_K_M · 21.7 GB | |
| 4 | DeepSeek R1 Distill Qwen 32BDeepSeek Q4_K_M · 24.3 GB · 234 t/s | 234 tok/s | Q4_K_M · 24.3 GB | |
| 5 | Hy3Tencent Q4_K_M · 182.3 GB · 53.7 t/s | 53.7 tok/s | Q4_K_M · 182.3 GB |
| # | Model | Speed | Vision Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 205.2 GB · 88.4 t/s | 88.4 tok/s | Q4_K_M · 205.2 GB | |
| 2 | Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 276 t/s | 276 tok/s | Q4_K_M · 17.9 GB | |
| 3 | Inkling SmallThinking Machines Q4_K_M · 162.7 GB · 116 t/s | 116 tok/s | Q4_K_M · 162.7 GB | |
| 4 | MiniMax M3MiniMax Q4_K_M · 258.0 GB · 69.1 t/s | 69.1 tok/s | Q4_K_M · 258.0 GB | |
| 5 | Qwen3.8 Flash NextAlibaba Q4_K_M · 109.0 GB · 57.7 t/s | 57.7 tok/s | Q4_K_M · 109.0 GB |
See all 433 models on the Instinct MI350X →Best quant that fits · speeds estimated on this card
The Instinct MI350X runs everything up to 120B MoE well. CDNA 4 generation, released 2025.
GenerationCDNA 4released 2025
Instinct MI350X strengths and weaknesses
Strengths
- Runs 120B MoE models: GPT OSS 120B at 126 tok/s in 70.8 GB
- Room for long context: Llama 3.3 70B Instruct at Q4 uses 51.3 GB, leaving 236.7 GB free
- Reads long prompts fast: A 4K-token prompt on MiMo V2.6 Flash RL starts answering in 242 ms
Weaknesses
- 1000 W under load: Plan a 2,000 W PSU; ≈$219 a year at 4 h a day
Output speed depends on memory bandwidth (8000 GB/s).
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB755tok/s
- 14BQwen3 14B · Q4_K_M · 11.9 GB505tok/s
- 27–31BQwen3.8 27B · Q4_K_M · 17.9 GB276tok/s
- 35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB116tok/s
- 70BLlama 3.3 70B Instruct · Q4_K_M · 51.3 GB103tok/s
- 120B MoEGPT OSS 120B · Q4_K_M · 70.8 GB126tok/s
Estimated speeds: Q4_K_M, 4K context, one user. How we estimate
All 1341 models on the Instinct MI350X
Featured onlyBest quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
| BF16 | 24.8 GB | 150.8 t/s | 131K | S97, Runs great | ||
| Q4_K_M | 21.5 GB | 490.8 t/s | 16K | S100, Runs great | ||
MythoMax L2 13BChat | Q4_K_M | 11.5 GB | 490.8 t/s | 4K | S100, Runs great | |
| Q4_K_M | 10.1 GB | 172.6 t/s | 8K | S98, Runs great | ||
Polyglot Ko 1.3BChat | Q4_K_M | 0.9 GB | 5052.6 t/s | 2K | S100, Runs great | |
Phi 3.5 Vision InstructVisionCode | Q4_K_M | 9.2 GB | 1337.0 t/s | 131K | S100, Runs great | |
| Q4_K_M | 49.3 GB | 107.6 t/s | 33K | S92, Runs great | ||
Nemotron Content Safety Reasoning 4BChatReasoning | Q4_K_M | 3.6 GB | 1481.5 t/s | 131K | S100, Runs great | |
Granite 3.1 2B InstructChatFunctions | Q4_K_M | 3.2 GB | 2412.1 t/s | 131K | S100, Runs great | |
| Q4_K_M | 7.3 GB | 890.5 t/s | 131K | S100, Runs great | ||
Llama 3.2 90B Vision InstructVisionFunctions | Q4_K_M | 63.5 GB | 82.1 t/s | — | S88, Runs great | |
| Q4_K_M | 2.7 GB | 1804.5 t/s | 2K | S100, Runs great | ||
LFM2 1.2BChatFunctions | Q4_K_M | 1.2 GB | 4486.0 t/s | 128K | S100, Runs great | |
DeepSeek v3 0324ChatFunctions | IQ3_XXS | 266.7 GB | 64.6 t/s | 164K | B62, Decent | |
| Q4_K_M | 0.4 GB | 12973.0 t/s | 1K | S100, Runs great |
Specs
- Architecture
- CDNA 4
- VRAM
- 288 GB HBM3e
- Memory bandwidth
- 8000.0 GB/s
- FP16 (dense)
- 2306.90 TFLOPS
- Stream processors
- 16,384
- TDP
- 1000 W
- Released
- 2025-06-12
Instinct MI350X FAQ
What is the largest model the Instinct MI350X can run?
Beam 1.0 501B A23B (501.3B) is the largest at 4-bit or better: at Q4_K_S it uses 285.2 GB of 288 GB, and runs at about 76.0 tok/s.
Can the Instinct MI350X run MiMo V2.6 Flash RL?
Yes. MiMo V2.6 Flash RL fits at Q4_K_M, using 186.5 GB of 288 GB, and it is the best model that runs well on this card.
How many tokens per second does the Instinct MI350X give?
About 87.1 tok/s on MiMo V2.6 Flash RL at Q4_K_M, with the first token after 0.2 s on a 4K-token prompt.
What power supply does the Instinct MI350X need?
Plan a 2000 W PSU (2× the 1000 W TDP).
Is 288 GB enough for coding and agents?
Yes for coding: MiMo V2.6 Flash RL, the best of the 25 coding models that fit, runs at about 87 tok/s.