NVIDIA · Blackwell

Best AI Models for NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition (96 GB)

Runs everything up to 120B MoE well.

96 GB GDDR71792 GB/s439 TFLOPS FP16300 WBlackwell 2025

300 W variant of the 96 GB RTX PRO 6000 — lower clocks than the 600 W edition, but two fit inside a 600 W budget for multi-GPU rigs.

What LLMs can the RTX PRO 6000 Blackwell Max-Q Workstation Edition run?

#	Model	Speed	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 57.1 t/s	57.1 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 250 t/s	250 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 244 t/s	244 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 57.1 t/s	57.1 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 213 t/s	213 tok/s	Q4_K_M · 22.0 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 250 t/s	250 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 244 t/s	244 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 57.1 t/s	57.1 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.5 27BAlibaba Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s	Q4_K_M · 17.9 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 250 t/s	250 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 244 t/s	244 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed	Math Score	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s		Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s		Q4_K_M · 17.9 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 213 t/s	213 tok/s		Q4_K_M · 22.0 GB
4	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 57.1 t/s	57.1 tok/s	overall	Q4_K_M · 21.7 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 244 t/s	244 tok/s		Q4_K_M · 19.3 GB

#	Model	Speed ↓	Quant · VRAM
1	Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 357 t/s	357 tok/s	Q4_K_M · 3.4 GB
2	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 282 t/s	282 tok/s	Q4_K_M · 13.1 GB
3	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 250 t/s	250 tok/s	Q4_K_M · 16.5 GB
4	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 244 t/s	244 tok/s	Q4_K_M · 19.3 GB
5	Qwen3 30B A3B Thinking 2507Alibaba Q4_K_M · 19.4 GB · 227 t/s	227 tok/s	Q4_K_M · 19.4 GB

#	Model	Speed	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 57.1 t/s	57.1 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 250 t/s	250 tok/s	Q4_K_M · 16.5 GB
5	JEV 27B VLautotrust Q4_K_M · 17.9 GB · 66.9 t/s	66.9 tok/s	Q4_K_M · 17.9 GB

See all 391 models on the RTX PRO 6000 Blackwell Max-Q Workstation Edition →Best quant that fits · speeds estimated on this card

About the RTX PRO 6000 Blackwell Max-Q Workstation Edition

The RTX PRO 6000 Blackwell Max-Q Workstation Edition runs everything up to 120B MoE well. Blackwell generation, released 2025.

GenerationBlackwellreleased 2025

RTX PRO 6000 Blackwell Max-Q Workstation Edition strengths and weaknesses

Strengths

Runs 120B MoE models: GPT OSS 120B at 202 tok/s in 70.8 GB
Room for long context: Llama 3.3 70B Instruct at Q4 uses 51.3 GB, leaving 44.7 GB free
Reads long prompts fast: A 4K-token prompt on Qwen3.8 27B starts answering in 1.2 s
FP8 and NVFP4 run natively: Blackwell takes FP8 and NVFP4 checkpoints in vLLM as published

RTX PRO 6000 Blackwell Max-Q Workstation Edition tokens per second by model size

Output speed depends on memory bandwidth (1792 GB/s).

9BQwen3.5 9B · Q4_K_M · 6.6 GB
183tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
123tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
66.9tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
213tok/s
70BLlama 3.3 70B Instruct · Q4_K_M · 51.3 GB
25.0tok/s
120B MoEGPT OSS 120B · Q4_K_M · 70.8 GB
202tok/s

Estimated speeds: Q4_K_M, 4K context, one user. How we estimate

All 1270 models on the RTX PRO 6000 Blackwell Max-Q Workstation Edition

Featured only

Best quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works

LLM models compatible with NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
SmolLM2 360M Instruct362MChat Q4_K_M·1941.3 t/s	Q4_K_M	0.8 GB1%	1941.3 t/s	8K	S100, Runs great
Llama3 OpenBioLLM 8B8BChat Q4_K_M·216.9 t/s	Q4_K_M	6.2 GB6%	216.9 t/s	8K	S100, Runs great
Seed OSS 36B Base36.2BChat Q4_K_M·52.3 t/s	Q4_K_M	24.1 GB25%	52.3 t/s	524K	A81, Runs well
Midm 2.0 Base Instruct11.5BChat Q4_K_M·152.7 t/s	Q4_K_M	10.4 GB11%	152.7 t/s	33K	S97, Runs great
Phi 4 Mini Flash Reasoning3.9BChatMathCodeReasoning Q4_K_M·394.8 t/s	Q4_K_M	5.3 GB6%	394.8 t/s	262K	S100, Runs great
Mistral 7B v0.27.2BChat Q4_K_M·237.2 t/s	Q4_K_M	6.8 GB7%	237.2 t/s	33K	S100, Runs great
TinyLLama V05MChat Q4_K_M·116480.0 t/s	Q4_K_M	0.0 GB0%	116480.0 t/s	2K	S100, Runs great
Olmo 3 7B Think7.3BChat Q4_K_M·202.6 t/s	Q4_K_M	8.4 GB9%	202.6 t/s	66K	S100, Runs great
Opt 1.3B1.3BChat FP16·407.3 t/s	FP16	2.9 GB3%	407.3 t/s	2K	S100, Runs great
HarmBench Llama 2 13B Cls13.0BChat Q4_K_M·119.0 t/s	Q4_K_M	9.8 GB10%	119.0 t/s	2K	S95, Runs great
Gpt2 Xl1.6BChat Q4_K_M·1098.9 t/s	Q4_K_M	1.1 GB1%	1098.9 t/s	1K	S100, Runs great
CodeLlama 70B Instruct HF69.0BChatCode Q4_K_M·27.5 t/s	Q4_K_M	43.0 GB45%	27.5 t/s	4K	A68, Runs well
Pythia 1.4B1.5BChat Q4_K_M·1164.8 t/s	Q4_K_M	1 GB1%	1164.8 t/s	2K	S100, Runs great
Zephyr 7B Beta7.2BChat Q4_K_M·237.2 t/s	Q4_K_M	6.8 GB7%	237.2 t/s	33K	S100, Runs great
SmolLM3 3B Base3.1BChat Q4_K_M·506.4 t/s	Q4_K_M	3.4 GB3%	506.4 t/s	66K	S100, Runs great

Rent NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition in the cloud

Hourly rate on demand for one RTX PRO 6000 Blackwell Max-Q Workstation Edition.

Vast.aiMarketplace
from$1.34/hr
Deploy

Buying one ($8,565) costs as much as ~6,400 hours of rental, about 27 months at 8 hours a day.

Cost per million tokens

Output-token cost on an RTX PRO 6000 Blackwell Max-Q Workstation Edition rented at $1.34/hr, one request at a time. Batch several users on the same card and the cost per token drops several-fold.

Model	Quant	Speed	Per 1M output
NVIDIA Nemotron 3.5 Lightning 30B A3B BF16	Q4_K_M	244 tok/s	$1.53$0.0089 input
GLM 4.7 Flash	Q4_K_M	196 tok/s	$1.90$0.017 input
Qwen3.8 27B	Q4_K_M	67 tok/s	$5.56$0.10 input
Muse Glimmer 30B	Q4_K_M	64 tok/s	$5.86$0.11 input
Gemma 4 31B IT	Q4_K_M	57 tok/s	$6.51$0.12 input

RTX PRO 6000 Blackwell Max-Q Workstation Edition specs, power and PSU

Specs

Architecture: Blackwell
VRAM: 96 GB GDDR7
Memory bandwidth: 1792.0 GB/s
FP16 (dense): 438.90 TFLOPS
CUDA cores: 24,064
Tensor cores: 752
Compute capability: 12.0
TDP: 300 W
Released: 2025-04-01
MSRP: $8,565

RTX PRO 6000 Blackwell Max-Q Workstation Edition FAQ

What is the largest model the RTX PRO 6000 Blackwell Max-Q Workstation Edition can run?

Mistral Medium 3.5 128B (127.7B) is the largest at 4-bit or better: at Q4_K_M it uses 82.8 GB of 96 GB, and runs at about 15.0 tok/s.

Can the RTX PRO 6000 Blackwell Max-Q Workstation Edition run Qwen3.8 27B?

Yes. Qwen3.8 27B fits at Q4_K_M, using 17.9 GB of 96 GB, and it is the best model that runs well on this card.

How many tokens per second does the RTX PRO 6000 Blackwell Max-Q Workstation Edition give?

About 66.9 tok/s on Qwen3.8 27B at Q4_K_M, with the first token after 1.2 s on a 4K-token prompt.

What power supply does the RTX PRO 6000 Blackwell Max-Q Workstation Edition need?

Plan a 600 W PSU (2× the 300 W TDP).

Is 96 GB enough for coding and agents?

Yes for coding: Qwen3.8 27B, the best of the 14 coding models that fit, runs at about 67 tok/s.

Best coding LLMs 48 GB VRAM GPUs with 96 GB or more RTX 5090 RTX 4090