AMD · CDNA 3

Best AI Models for AMD Instinct MI325X (256 GB)

Runs everything up to 120B MoE well.

256 GB HBM3e6000 GB/s1307 TFLOPS FP161000 WCDNA 3 2024

Datacenter accelerator — cloud/server only; ROCm software stack is less mature than CUDA. Listed for comparison.

What LLMs can the Instinct MI325X run?

#	Model	Speed	Quant · VRAM
1	MiMo V2.6 Flash RLXiaomiHow to run → Q4_K_M · 186.5 GB · 82.6 t/s	82.6 tok/s	Q4_K_M · 186.5 GB
2	DeepSeek V4 Flash 0731DeepSeek Q4_K_M · 183.0 GB · 75.9 t/s	75.9 tok/s	Q4_K_M · 183.0 GB
3	GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 82.8 t/s	82.8 tok/s	Q4_K_M · 205.2 GB
4	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed	Quant · VRAM
1	MiMo V2.6 Flash RLXiaomiHow to run → Q4_K_M · 186.5 GB · 82.6 t/s	82.6 tok/s	Q4_K_M · 186.5 GB
2	DeepSeek V4 Flash 0731DeepSeek Q4_K_M · 183.0 GB · 75.9 t/s	75.9 tok/s	Q4_K_M · 183.0 GB
3	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
4	GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 82.8 t/s	82.8 tok/s	Q4_K_M · 205.2 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed	Quant · VRAM
1	DeepSeek V4 Flash 0731DeepSeekHow to run → Q4_K_M · 183.0 GB · 75.9 t/s	75.9 tok/s	Q4_K_M · 183.0 GB
2	GLM 5.3 FlashZ.ai Q4_K_M · 205.2 GB · 82.8 t/s	82.8 tok/s	Q4_K_M · 205.2 GB
3	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
4	Inkling SmallThinking Machines Q4_K_M · 162.7 GB · 115 t/s	115 tok/s	Q4_K_M · 162.7 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed	Quant · VRAM
1	MiMo V2.6 Flash RLXiaomiHow to run → Q4_K_M · 186.5 GB · 82.6 t/s	82.6 tok/s	Q4_K_M · 186.5 GB
2	DeepSeek V4 Flash 0731DeepSeek Q4_K_M · 183.0 GB · 75.9 t/s	75.9 tok/s	Q4_K_M · 183.0 GB
3	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
4	DeepSeek V4 FlashDeepSeek Q4_K_M · 175.1 GB · 91.3 t/s	91.3 tok/s	Q4_K_M · 175.1 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed ↓	Quant · VRAM
1	Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 1104 t/s	1104 tok/s	Q4_K_M · 3.4 GB
2	Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 566 t/s	566 tok/s	Q4_K_M · 6.6 GB
3	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 181 t/s	181 tok/s	Q4_K_M · 13.1 GB
4	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 177 t/s	177 tok/s	Q4_K_M · 21.7 GB
5	Hy3Tencent Q4_K_M · 182.3 GB · 51.3 t/s	51.3 tok/s	Q4_K_M · 182.3 GB

#	Model	Speed	Quant · VRAM
1	GLM 5.3 FlashZ.aiHow to run → Q4_K_M · 205.2 GB · 82.8 t/s	82.8 tok/s	Q4_K_M · 205.2 GB
2	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
3	Inkling SmallThinking Machines Q4_K_M · 162.7 GB · 115 t/s	115 tok/s	Q4_K_M · 162.7 GB
4	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB
5	JEV 27B VLautotrust Q4_K_M · 17.9 GB · 207 t/s	207 tok/s	Q4_K_M · 17.9 GB

See all 425 models on the Instinct MI325X →Best quant that fits · speeds estimated on this card

About the Instinct MI325X

The Instinct MI325X runs everything up to 120B MoE well. CDNA 3 generation, released 2024.

GenerationCDNA 3released 2024

Instinct MI325X strengths and weaknesses

Strengths

Runs 120B MoE models: GPT OSS 120B at 123 tok/s in 70.8 GB
Room for long context: Llama 3.3 70B Instruct at Q4 uses 51.3 GB, leaving 204.7 GB free
Reads long prompts fast: A 4K-token prompt on MiMo V2.6 Flash RL starts answering in 427 ms

Weaknesses

1000 W under load: Plan a 2,000 W PSU; ≈$219 a year at 4 h a day

Instinct MI325X tokens per second by model size

Output speed depends on memory bandwidth (6000 GB/s).

9BQwen3.5 9B · Q4_K_M · 6.6 GB
566tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
379tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
207tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
114tok/s
70BLlama 3.3 70B Instruct · Q4_K_M · 51.3 GB
77.3tok/s
120B MoEGPT OSS 120B · Q4_K_M · 70.8 GB
123tok/s

Estimated speeds: Q4_K_M, 4K context, one user. How we estimate

All 1329 models on the Instinct MI325X

Featured only

Best quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works

LLM models compatible with AMD Instinct MI325X — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
Nidum Gemma 2B Uncensored2.5BChat Q4_K_M·1956.5 t/s	Q4_K_M	1.9 GB1%	1956.5 t/s	8K	S100, Runs great
Qwen2.5 32B Instruct32.8BChatFunctions Q4_K_M·175.6 t/s	Q4_K_M	24.3 GB9%	175.6 t/s	33K	S99, Runs great
Qwen3 42B A3B 2507 Thinking Abliterated Uncensored TOTAL RECALL v2 Medium MASTER CODER42.4BChatCodeReasoning Q4_K_M·70.7 t/s	Q4_K_M	26.8 GB10%	70.7 t/s	262K	S86, Runs great
Llama3 8B Chinese Chat8.0BChat Q4_K_M·667.9 t/s	Q4_K_M	6.2 GB2%	667.9 t/s	8K	S100, Runs great
Qwen3.6 27B MTPLX Optimized26.9BChat Q4_K_M·213.3 t/s	Q4_K_M	17.3 GB7%	213.3 t/s	262K	S100, Runs great
Vicuna 13B V1.313BChat Q4_K_M·419.6 t/s	Q4_K_M	8.6 GB3%	419.6 t/s	2K	S100, Runs great
Finance Llama3 8B8.0BChat Q4_K_M·667.9 t/s	Q4_K_M	6.2 GB2%	667.9 t/s	8K	S100, Runs great
GLM 5.1753.9BChatFunctions IQ2_S·51.7 t/s	IQ2_S	237.4 GB93%	51.7 t/s	203K	B61, Decent
MobileLLaMA 1.4B Chat1.4BChat Q4_K_M·2337.7 t/s	Q4_K_M	1.5 GB1%	2337.7 t/s	2K	S100, Runs great
GPT OSS Safeguard 20B21.5BChatFunctions Q4_K_M·177.9 t/s	Q4_K_M	13.5 GB5%	177.9 t/s	131K	S99, Runs great
TinyStories 1M1MChat BF16·360000.0 t/s	BF16	0.0 GB0%	360000.0 t/s	2K	S100, Runs great
Phi 4 Mini Instruct3.8BChatCodeFunctions Q4_K_M·1254.4 t/s	Q4_K_M	4.8 GB2%	1254.4 t/s	131K	S100, Runs great
Step 3.5 Flash Base197.8BChat Q4_K_M·94.9 t/s	Q4_K_M	136.8 GB53%	94.9 t/s	33K	S90, Runs great
Phi 3.5 MoE Instruct41.9BChatCode Q4_K_M·132.8 t/s	Q4_K_M	27.6 GB11%	132.8 t/s	131K	S95, Runs great
HarmBench Llama 2 13B Cls13.0BChat Q4_K_M·367.7 t/s	Q4_K_M	9.8 GB4%	367.7 t/s	2K	S100, Runs great

Instinct MI325X specs, power and PSU

Specs

Architecture: CDNA 3
VRAM: 256 GB HBM3e
Memory bandwidth: 6000.0 GB/s
FP16 (dense): 1307.40 TFLOPS
Stream processors: 19,456
TDP: 1000 W
Released: 2024-10-10

Instinct MI325X FAQ

What is the largest model the Instinct MI325X can run?

Beam 1.0 501B A23B (501.3B) is the largest at 4-bit or better: at Q4_0 it uses 253.8 GB of 256 GB, and runs at about 73.2 tok/s.

Can the Instinct MI325X run MiMo V2.6 Flash RL?

Yes. MiMo V2.6 Flash RL fits at Q4_K_M, using 186.5 GB of 256 GB, and it is the best model that runs well on this card.

How many tokens per second does the Instinct MI325X give?

About 82.6 tok/s on MiMo V2.6 Flash RL at Q4_K_M, with the first token after 0.4 s on a 4K-token prompt.

What power supply does the Instinct MI325X need?

Plan a 2000 W PSU (2× the 1000 W TDP).

Is 256 GB enough for coding and agents?

Yes for coding: MiMo V2.6 Flash RL, the best of the 24 coding models that fit, runs at about 83 tok/s.

Best coding LLMs 48 GB VRAM GPUs with 256 GB or more RTX 5090 RTX 4090 RTX 3090