Best AI Models for AMD Instinct MI355X Platform (8x MI355X) (2.3 TB)

The same chip from other makers first, then machines and GPUs people weigh against it, each with the one reason it is worth a look. Speeds on Qwen3.6 35B A3B.

Output speed depends on memory bandwidth: each model runs on as few of the 8 GPUs as hold it, at 8 TB/s per GPU. Grey rows need more than the 2.3 TB available.

Best quant that fits in 2304 GB, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works

LLM models compatible with AMD Instinct MI355X Platform (8x MI355X) — ranked by Score, recency and Fit

Model	Quant	VRAM	Speed	Context	Score	Fit
Ornith 1.5 9B OBLITERATED9.7BChatFunctions Q4_K_M·754.7 t/s	Q4_K_M	6.6 GB0%	754.7 t/s	262K	Traction 164.0 of 500	S100, Runs great
Qwen3.5 2B2.3BVisionFunctions Q4_K_M·2711.9 t/s	Q4_K_M	1.9 GB0%	2711.9 t/s	262K	Traction 288.0 of 500	S100, Runs great
ZDTaichu5.0 9B9.8BVisionFunctions Q4_K_M·743.0 t/s	Q4_K_M	9.2 GB0%	743.0 t/s	—	Traction 58.0 of 500	S100, Runs great
Huihui Qwen3.8 27B Abliterated27.8BVisionFunctions Q4_K_M·275.5 t/s	Q4_K_M	17.9 GB1%	275.5 t/s	262K	Traction 134.5 of 500	S100, Runs great
Granite Switch 4.1 8B8.8BChatFunctions Q4_K_M·810.8 t/s	Q4_K_M	8.4 GB0%	810.8 t/s	131K	Traction 17.5 of 500	S100, Runs great
Ornith 1.5 35B A3B36.0BChatFunctions Q4_K_M·116.2 t/s	Q4_K_M	22.0 GB1%	116.2 t/s	262K	Traction 182.0 of 500	S94, Runs great
Qwen3.8 2.4T A95B2446.2BChatFunctions Q4_K_M·49.3 t/s	Q4_K_M	1468.8 GB64%	49.3 t/s	262K	Traction 92.0 of 500	A79, Runs well
Ornith 1.5 9B9.7BChatFunctions Q4_K_M·754.7 t/s	Q4_K_M	6.6 GB0%	754.7 t/s	262K	Traction 199.5 of 500	S100, Runs great
LensVLM 9B9.4BVisionFunctions Q4_K_M·772.9 t/s	Q4_K_M	6.5 GB0%	772.9 t/s	262K	Traction 34.5 of 500	S100, Runs great
Phi 414.7BChatMath</>Code Q4_K_M·504.2 t/s	Q4_K_M	12.5 GB1%	504.2 t/s	16K	83.9	S100, Runs great
Granite Switch 4.1 30B30.0BChatFunctions Q4_K_M·254.6 t/s	Q4_K_M	22.7 GB1%	254.6 t/s	131K	Traction 16.0 of 500	S100, Runs great
DeepSeek R1 0528684.5BChatReasoningFunctions Q4_K_M·67.9 t/s	Q4_K_M	412.2 GB18%	67.9 t/s	164K	103.7	S85, Runs great
MiniMax M2.5228.7BChatFunctions Q4_K_M·72.5 t/s	Q4_K_M	139.6 GB6%	72.5 t/s	197K	111.8	S86, Runs great
Intern Decision 4B4.5BVisionFunctions Q4_K_M·1504.7 t/s	Q4_K_M	3.4 GB0%	1504.7 t/s	262K	Traction 23.0 of 500	S100, Runs great
Kimi K2 Thinking1026.4BChatFunctions Q4_K_M·75.6 t/s	Q4_K_M	617.3 GB27%	75.6 t/s	262K	108.2	S86, Runs great