AMD · 8x Instinct MI355X (OAM, UBB 2.0) · Server
Best AI Models for AMD Instinct MI355X Platform (8x MI355X) (2.3 TB)
Runs models up to 2.8T parameters, like Kimi K3.1 Pro.
2.3 TB HBM64 TB/s20.1 PFLOPS FP162025
| # | Model | Speed | llmrun Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Kimi K3.1 ProMoonshot AIHow to run → Q4_K_M · 1712.1 GB · 49.4 t/s | 49.4 tok/s | Q4_K_M · 1712.1 GB | |
| 2 | Inkling 2 SmallThinking Machines Q4_K_M · 359.1 GB · 93.8 t/s | 93.8 tok/s | Q4_K_M · 359.1 GB | |
| 3 | DeepSeek V4.1 Flash 1017DeepSeek Q4_K_M · 458.4 GB · 111 t/s | 111 tok/s | Q4_K_M · 458.4 GB | |
| 4 | MiMo V2.6 Pro RLXiaomi Q4_K_M · 614.1 GB · 65.3 t/s | 65.3 tok/s | Q4_K_M · 614.1 GB | |
| 5 | Muse Glimmer 30BMeta Q4_K_M · 19.6 GB · 262 t/s | 262 tok/s | Q4_K_M · 19.6 GB |
| # | Model | Speed | Coding Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Kimi K3.1 ProMoonshot AIHow to run → Q4_K_M · 1712.1 GB · 49.4 t/s | 49.4 tok/s | Q4_K_M · 1712.1 GB | |
| 2 | DeepSeek V4.1 Flash 1017DeepSeek Q4_K_M · 458.4 GB · 111 t/s | 111 tok/s | Q4_K_M · 458.4 GB | |
| 3 | Kimi K3Moonshot AI Q4_K_M · 1670.0 GB · 49.4 t/s | 49.4 tok/s | Q4_K_M · 1670.0 GB | |
| 4 | GLM 5.3Z.ai Q4_K_M · 453.8 GB · 54.8 t/s | 54.8 tok/s | Q4_K_M · 453.8 GB | |
| 5 | Muse Glimmer 30BMeta Q4_K_M · 19.6 GB · 262 t/s | 262 tok/s | Q4_K_M · 19.6 GB |
| # | Model | Speed | Reasoning Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Q4_K_M · 458.4 GB · 111 t/s | 111 tok/s | Q4_K_M · 458.4 GB | |
| 2 | Inkling 2 SmallThinking Machines Q4_K_M · 359.1 GB · 93.8 t/s | 93.8 tok/s | Q4_K_M · 359.1 GB | |
| 3 | Kimi K3.1 ProMoonshot AI Q4_K_M · 1712.1 GB · 49.4 t/s | 49.4 tok/s | Q4_K_M · 1712.1 GB | |
| 4 | DeepSeek V4.1 FlashDeepSeek Q4_K_M · 458.4 GB · 111 t/s | 111 tok/s | Q4_K_M · 458.4 GB | |
| 5 | Muse Glimmer 30BMeta Q4_K_M · 19.6 GB · 262 t/s | 262 tok/s | Q4_K_M · 19.6 GB |
| # | Model | Speed | Math Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Kimi K3Moonshot AIHow to run → Q4_K_M · 1670.0 GB · 49.4 t/s | 49.4 tok/s | Q4_K_M · 1670.0 GB | |
| 2 | DeepSeek V4.1 Flash 1017DeepSeek Q4_K_M · 458.4 GB · 111 t/s | 111 tok/s | Q4_K_M · 458.4 GB | |
| 3 | DeepSeek V4.1 FlashDeepSeek Q4_K_M · 458.4 GB · 111 t/s | 111 tok/s | Q4_K_M · 458.4 GB | |
| 4 | GLM 5.3Z.ai Q4_K_M · 453.8 GB · 54.8 t/s | 54.8 tok/s | Q4_K_M · 453.8 GB | |
| 5 | Muse Glimmer 30BMeta Q4_K_M · 19.6 GB · 262 t/s | 262 tok/s | Q4_K_M · 19.6 GB |
| # | Model | Speed ↓ | llmrun Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 1472 t/s | 1472 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 755 t/s | 755 tok/s | Q4_K_M · 6.6 GB | |
| 3 | Muse Glimmer 30BMeta Q4_K_M · 19.6 GB · 262 t/s | 262 tok/s | Q4_K_M · 19.6 GB | |
| 4 | Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 235 t/s | 235 tok/s | Q4_K_M · 21.7 GB | |
| 5 | DeepSeek R1 Distill Qwen 32BDeepSeek Q4_K_M · 24.3 GB · 234 t/s | 234 tok/s | Q4_K_M · 24.3 GB |
| # | Model | Speed | Agents Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Q4_K_M · 458.4 GB · 111 t/s | 111 tok/s | Q4_K_M · 458.4 GB | |
| 2 | Kimi K3.1 ProMoonshot AI Q4_K_M · 1712.1 GB · 49.4 t/s | 49.4 tok/s | Q4_K_M · 1712.1 GB | |
| 3 | GLM 5.3Z.ai Q4_K_M · 453.8 GB · 54.8 t/s | 54.8 tok/s | Q4_K_M · 453.8 GB | |
| 4 | MiMo V2.6 Pro RLXiaomi Q4_K_M · 614.1 GB · 65.3 t/s | 65.3 tok/s | Q4_K_M · 614.1 GB | |
| 5 | Muse Glimmer 30BMeta Q4_K_M · 19.6 GB · 262 t/s | 262 tok/s | Q4_K_M · 19.6 GB |
See all 454 models on the AMD Instinct MI355X Platform (8x MI355X) →Bars = traction for models without a Score yet
The AMD Instinct MI355X Platform (8x MI355X) runs models up to 2.8T parameters, like Kimi K3.1 Pro. A 4K-token prompt waits 200 ms before the first word on Kimi K3.1 Pro. 8x Instinct MI355X (OAM, UBB 2.0) (CDNA 4), released 2025.
For models2.3 TBall of it
Chip8x Instinct MI355X (OAM, UBB 2.0)released 2025
AMD Instinct MI355X Platform (8x MI355X) strengths and weaknesses
Strengths
- Runs 120B MoE models: GPT OSS 120B at 126 tok/s in 70.8 GB
- Room for long context: Llama 3.3 70B Instruct at Q4 uses 51.3 GB, leaving 2.3 TB free
- Reads long prompts fast: A 32K-token prompt on Kimi K3.1 Pro starts answering in 1.6 s
Memory for models
2.3 TB of 2.3 TB120B MoE70.8 GB2780B1712.1 GB
120B MoE70.8 GB2780B1712.1 GB
For models 2.3 TBModel size at Q4
Everything on the size ladder fits. Kimi K3.1 Pro and MiMo V2.6 Flash RL together use 1.9 TB and leave 405.4 GB free.
The same chip from other makers first, then machines and GPUs people weigh against it, each with the one reason it is worth a look. Speeds on Qwen3.6 35B A3B.
Step down to 508.5 GBMac Studio M5 Ultra (512 GB)Apple · 508.5 GB for models · M5 Ultra
▼ 24% speed▼ 1.8 TBabout $22,299
Step down to 508.5 GBMac Studio M3 Ultra (512 GB)Apple · 508.5 GB for models · M3 Ultra▼ 34% speed▼ 1.8 TBabout $9,499
Step down to 188.5 GBMac Studio M2 Ultra (192 GB)Apple · 188.5 GB for models · M2 Ultra▼ 35% speed▼ 2.1 TBabout $5,599
Step down to 124.5 GBMac Studio M4 Max (128 GB)Apple · 124.5 GB for models · M4 Max▼ 45% speed▼ 2.2 TBabout $3,499
Step down to 24 GBRTX 4090NVIDIA · 24 GB VRAM · GPU▲ 30% speed▼ 2.3 TBabout $1,599rent $0.34/hr
1.8× faster if your models fit in 32 GBRTX 5090NVIDIA · 32 GB VRAM · GPU▲ 84% speed▼ 2.3 TBabout $5,502rent $0.50/hr
Output speed depends on memory bandwidth: each model runs on as few of the 8 GPUs as hold it, at 8 TB/s per GPU. Grey rows need more than the 2.3 TB available.
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB755tok/s
- 14BQwen3 14B · Q4_K_M · 11.9 GB505tok/s
- 27–31BQwen3.8 27B · Q4_K_M · 17.9 GB276tok/s
- 35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB116tok/s
- 70BLlama 3.3 70B Instruct · Q4_K_M · 51.3 GB103tok/s
- 120B MoEGPT OSS 120B · Q4_K_M · 70.8 GB126tok/s
Estimated speeds: Q4_K_M, 4K context, one user. How we estimate
All 454 models on the AMD Instinct MI355X Platform (8x MI355X)
Featured onlyBest quant that fits in 2304 GB, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
Qwen2.5 VL 32B InstructVision | Q4_K_M | 24.7 GB | 229.6 t/s | 128K | S100, Runs great | |
Llama 3.1 405B InstructChatFunctions | Q4_K_M | 275.6 GB | 17.9 t/s | — | B58, Decent | |
Mixtral 8x22B Instruct v0.1ChatFunctions | Q4_K_M | 88.4 GB | 62.0 t/s | 66K | A84, Runs well | |
C4ai Command R 08 2024ChatFunctions | Q4_K_M | 25.1 GB | 225.1 t/s | — | S100, Runs great | |
DeepSeek v3 0324ChatFunctions | Q4_K_M | 412.2 GB | 67.9 t/s | 164K | S85, Runs great | |
Llama 3.1 8B InstructChatFunctions | Q4_K_M | 7.9 GB | 905.7 t/s | 131K | S100, Runs great | |
Qwen2.5 Omni 3BChat | Q4_K_M | 6.0 GB | 1315.1 t/s | — | S100, Runs great | |
| BF16 | 573.0 GB | 85.8 t/s | 1049K | S88, Runs great | ||
DeepSeek v3ChatFunctions | Q4_K_M | 412.1 GB | 67.9 t/s | 164K | S85, Runs great | |
| Q4_K_M | 1.0 GB | 4752.5 t/s | 2K | S100, Runs great | ||
Qwen2.5 Coder 32B InstructChatCodeFunctions | Q4_K_M | 24.3 GB | 234.1 t/s | 33K | S100, Runs great | |
Gemma 3 27B ITVision | Q4_K_M | 21.7 GB | 265.0 t/s | 131K | S100, Runs great | |
Gemma 2 9B ITChat | Q4_K_M | 7.3 GB | 786.9 t/s | 8K | S100, Runs great | |
Qwen3 VL 2B InstructVisionFunctions | Q4_K_M | 3.5 GB | 2651.9 t/s | 262K | S100, Runs great | |
Llava V1.6 Mistral 7B HFVision | FP16 | 19.2 GB | 288.3 t/s | 33K | S100, Runs great |
Specs
- Chip
- 8x Instinct MI355X (OAM, UBB 2.0) · AMD Instinct MI355X
- GPU cores
- 131,072
- Memory
- 2.3 TB
- For models
- 2.3 TB
- Memory bandwidth
- 64 TB/s
- FP16 (dense)
- 20.1 PFLOPS
- Released
- 2025-06-12
AMD Instinct MI355X Platform (8x MI355X) FAQ
How much memory is available for AI on the AMD Instinct MI355X Platform (8x MI355X)?
All 2.3 TB can hold model weights.
What is the largest model the AMD Instinct MI355X Platform (8x MI355X) can run?
Kimi K3.1 Pro (2780.1B) is the largest at 4-bit or better: at Q4_K_M it uses 1712.1 GB, 74% of the available memory, and runs at about 49.4 tok/s.
How long does the AMD Instinct MI355X Platform (8x MI355X) take to read a long prompt?
On Kimi K3.1 Pro, about 200 ms for a 4K-token prompt and 1.6 s for a 32K-token document before the first word appears.