AMD · CDNA 3

Best AI Models for AMD Instinct MI325X (256 GB)

Runs everything up to 120B MoE well.

256 GB HBM3e6000 GB/s1307 TFLOPS FP161000 WCDNA 3 2024

Datacenter accelerator — cloud/server only; ROCm software stack is less mature than CUDA. Listed for comparison.

What LLMs can the Instinct MI325X run?

#	Model	Speed	Quant · VRAM
1	MiMo V2.6 Flash RLXiaomiHow to run → Q4_K_M · 186.5 GB · 82.6 t/s	82.6 tok/s	Q4_K_M · 186.5 GB
2	DeepSeek V4 Flash 0731DeepSeek Q4_K_M · 183.0 GB · 75.9 t/s	75.9 tok/s	Q4_K_M · 183.0 GB
3	GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 82.8 t/s	82.8 tok/s	Q4_K_M · 205.2 GB
4	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed	Quant · VRAM
1	MiMo V2.6 Flash RLXiaomiHow to run → Q4_K_M · 186.5 GB · 82.6 t/s	82.6 tok/s	Q4_K_M · 186.5 GB
2	DeepSeek V4 Flash 0731DeepSeek Q4_K_M · 183.0 GB · 75.9 t/s	75.9 tok/s	Q4_K_M · 183.0 GB
3	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
4	GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 82.8 t/s	82.8 tok/s	Q4_K_M · 205.2 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed	Quant · VRAM
1	DeepSeek V4 Flash 0731DeepSeekHow to run → Q4_K_M · 183.0 GB · 75.9 t/s	75.9 tok/s	Q4_K_M · 183.0 GB
2	GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 82.8 t/s	82.8 tok/s	Q4_K_M · 205.2 GB
3	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
4	Inkling SmallThinking Machines Q4_K_M · 162.7 GB · 115 t/s	115 tok/s	Q4_K_M · 162.7 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed	Quant · VRAM
1	MiMo V2.6 Flash RLXiaomiHow to run → Q4_K_M · 186.5 GB · 82.6 t/s	82.6 tok/s	Q4_K_M · 186.5 GB
2	DeepSeek V4 Flash 0731DeepSeek Q4_K_M · 183.0 GB · 75.9 t/s	75.9 tok/s	Q4_K_M · 183.0 GB
3	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
4	DeepSeek V4 FlashDeepSeek Q4_K_M · 175.1 GB · 91.3 t/s	91.3 tok/s	Q4_K_M · 175.1 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed ↓	Quant · VRAM
1	Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 1104 t/s	1104 tok/s	Q4_K_M · 3.4 GB
2	Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 566 t/s	566 tok/s	Q4_K_M · 6.6 GB
3	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 181 t/s	181 tok/s	Q4_K_M · 13.1 GB
4	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 177 t/s	177 tok/s	Q4_K_M · 21.7 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed	Quant · VRAM
1	GLM 5.3 FlashZ.aiHow to run → Q4_K_M · 205.2 GB · 82.8 t/s	82.8 tok/s	Q4_K_M · 205.2 GB
2	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
3	Inkling SmallThinking Machines Q4_K_M · 162.7 GB · 115 t/s	115 tok/s	Q4_K_M · 162.7 GB
4	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
5	JEV 27B VLautotrust Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB

See all 425 models on the Instinct MI325X →Best quant that fits · speeds estimated on this card

About the Instinct MI325X

The Instinct MI325X runs everything up to 120B MoE well. CDNA 3 generation, released 2024.

GenerationCDNA 3released 2024

Instinct MI325X strengths and weaknesses

Strengths

Runs 120B MoE models: GPT OSS 120B at 123 tok/s in 70.8 GB
Room for long context: Llama 3.3 70B Instruct at Q4 uses 51.3 GB, leaving 204.7 GB free
Reads long prompts fast: A 4K-token prompt on MiMo V2.6 Flash RL starts answering in 427 ms

Weaknesses

1000 W under load: Plan a 2,000 W PSU; ≈$219 a year at 4 h a day

Instinct MI325X tokens per second by model size

Output speed depends on memory bandwidth (6000 GB/s).

9BQwen3.5 9B · Q4_K_M · 6.6 GB
566tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
379tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
207tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
114tok/s
70BLlama 3.3 70B Instruct · Q4_K_M · 51.3 GB
77.3tok/s
120B MoEGPT OSS 120B · Q4_K_M · 70.8 GB
123tok/s

Estimated speeds: Q4_K_M, 4K context, one user. How we estimate

All 1329 models on the Instinct MI325X

Featured only

Best quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works

LLM models compatible with AMD Instinct MI325X — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
DeepSeek R1 Distill Qwen 14B14.8BChatReasoningFunctions Q4_K_M·376.6 t/s	Q4_K_M	12.4 GB5%	376.6 t/s	131K	S100, Runs great
PicoMistral 23M24MChat Q4_K_M·11250.0 t/s	Q4_K_M	0.3 GB0%	11250.0 t/s	1K	S100, Runs great
Ouro Hybrid 1.4B1.5BChat BF16·965.1 t/s	BF16	6.5 GB3%	965.1 t/s	66K	S100, Runs great
Qwen3 4B4.0BChatFunctions Q4_K_M·1241.4 t/s	Q4_K_M	4.2 GB2%	1241.4 t/s	41K	S100, Runs great
Supra Mini V6 1M1MChat Q4_K_M·12000.0 t/s	Q4_K_M	0.3 GB0%	12000.0 t/s	1K	S100, Runs great
Qwen3.5 4B MiniFantasy MTP4.7BChatRoleplay FP16·367.7 t/s	FP16	10.0 GB4%	367.7 t/s	262K	S100, Runs great
Qwen3.5 35B A3B Claude 4.6 Opus Reasoning Distilled36.0BChatReasoning BF16·93.9 t/s	BF16	72.4 GB28%	93.9 t/s	262K	S90, Runs great
DeepSeek V4 Flash 162B92.2BChat Q4_K_M·64.7 t/s	Q4_K_M	55.8 GB22%	64.7 t/s	1049K	A84, Runs well
Hunyuan A13B Instruct80.4BChat Q4_K_M·117.0 t/s	Q4_K_M	50.7 GB20%	117.0 t/s	33K	S94, Runs great
GPT S 1.4M1MChat BF16·11612.9 t/s	BF16	0.3 GB0%	11612.9 t/s	0K	S100, Runs great
Ethereal Stardust 12B12.2BChatRoleplay Q4_K_M·446.1 t/s	Q4_K_M	11.0 GB4%	446.1 t/s	131K	S100, Runs great
OvisOCR2853MVisionFunctions Q4_K_M·4186.0 t/s	Q4_K_M	0.9 GB0%	4186.0 t/s	262K	S100, Runs great
Devstral Small 2 24B Instruct 251224.0BChatFunctions Q4_K_M·237.9 t/s	Q4_K_M	18.1 GB7%	237.9 t/s	393K	S100, Runs great
Qwen3 4B Hindi Instruct v24.0BChat Q4_K_M·1241.4 t/s	Q4_K_M	4.2 GB2%	1241.4 t/s	262K	S100, Runs great
Josiefied Qwen3.5 0.8B Gabliterated V1853MChat BF16·1747.6 t/s	BF16	2.1 GB1%	1747.6 t/s	262K	S100, Runs great

Instinct MI325X specs, power and PSU

Specs

Architecture: CDNA 3
VRAM: 256 GB HBM3e
Memory bandwidth: 6000.0 GB/s
FP16 (dense): 1307.40 TFLOPS
Stream processors: 19,456
TDP: 1000 W
Released: 2024-10-10

Instinct MI325X FAQ

What is the largest model the Instinct MI325X can run?

Beam 1.0 501B A23B (501.3B) is the largest at 4-bit or better: at Q4_0 it uses 253.8 GB of 256 GB, and runs at about 73.2 tok/s.

Can the Instinct MI325X run MiMo V2.6 Flash RL?

Yes. MiMo V2.6 Flash RL fits at Q4_K_M, using 186.5 GB of 256 GB, and it is the best model that runs well on this card.

How many tokens per second does the Instinct MI325X give?

About 82.6 tok/s on MiMo V2.6 Flash RL at Q4_K_M, with the first token after 0.4 s on a 4K-token prompt.

What power supply does the Instinct MI325X need?

Plan a 2000 W PSU (2× the 1000 W TDP).

Is 256 GB enough for coding and agents?

Yes for coding: MiMo V2.6 Flash RL, the best of the 24 coding models that fit, runs at about 83 tok/s.

Best coding LLMs 48 GB VRAM GPUs with 256 GB or more RTX 5090 RTX 4090 RTX 3090