Can NVIDIA GeForce RTX 4090 run DeepSeek R1 Distill Qwen 32B?

Yes, at Q5_K_S (23.4 GB) or lower. Higher quantizations like Q5_K_M (24.2 GB) exceed the NVIDIA GeForce RTX 4090's 24 GB.

What's the best quantization for DeepSeek R1 Distill Qwen 32B?

For DeepSeek R1 Distill Qwen 32B, Q4_K_M (20.5 GB) offers the best balance of quality and VRAM usage. Q4_K_L (20.9 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 9.8 GB.

Can I run DeepSeek R1 Distill Qwen 32B on a Mac?

DeepSeek R1 Distill Qwen 32B requires at least 9.8 GB at IQ2_XXS, which exceeds the unified memory of most consumer Macs. You would need a Mac Studio or Mac Pro with a high-memory configuration.

Can I run DeepSeek R1 Distill Qwen 32B locally?

Yes — DeepSeek R1 Distill Qwen 32B can run locally on consumer hardware. At Q4_K_M quantization it needs 20.5 GB of VRAM. Popular tools include Ollama, LM Studio, and llama.cpp.

How fast is DeepSeek R1 Distill Qwen 32B?

At Q4_K_M, DeepSeek R1 Distill Qwen 32B can reach ~142 tok/s on AMD Instinct MI300X. On NVIDIA GeForce RTX 4090: ~32 tok/s. Speed depends mainly on GPU memory bandwidth. Real-world results typically within ±20%.

What's the download size of DeepSeek R1 Distill Qwen 32B?

At Q4_K_M, the download is about 19.66 GB. The full-precision Q8_0 version is 32.76 GB. The smallest option (IQ2_XXS) is 9.01 GB.

DeepSeek·Qwen·Qwen2ForCausalLM

DeepSeek R1 Distill Qwen 32B — Hardware Requirements & GPU Compatibility

ChatReasoning

DeepSeek R1 Distill Qwen 32B takes the reasoning capabilities developed in the full 684.5B R1 model and distills them into the 32.8 billion parameter Qwen 2.5 architecture. The result is a dense model that punches well above its weight class on math, science, and coding reasoning tasks, often matching models two to three times its size. At around 32.8 billion parameters, this model fits comfortably on a single high-end consumer GPU when quantized to 4-bit precision, making it one of the most capable reasoning models you can run on a desktop workstation.

938.1K downloads 1.5K likesFeb 2025131K context

Specifications

Publisher: DeepSeek
Family: Qwen
Parameters: 32.8B
Architecture: Qwen2ForCausalLM
Context Length: 131,072 tokens
Vocabulary Size: 152,064
Release Date: 2025-02-24
License: MIT

Get Started

HuggingFace

deepseek-ai/DeepSeek-R1-Distill-Qwen-32B

How Much VRAM Does DeepSeek R1 Distill Qwen 32B Need?

Select a quantization to see compatible GPUs below.

Quantization	Bits	VRAM	+ Context	File Size	Quality
IQ2_XXS	2.20	9.8 GB	43.7 GB	9.01 GB	Importance-weighted 2-bit, extreme compression — significant quality loss
IQ2_XS	2.40	10.7 GB	44.5 GB	9.83 GB	Importance-weighted 2-bit, extra small
IQ2_S	2.50	11.1 GB	44.9 GB	10.24 GB	Importance-weighted 2-bit, small
IQ2_M	2.70	11.9 GB	45.7 GB	11.06 GB	Importance-weighted 2-bit, medium
IQ3_XS	3.30	14.3 GB	48.2 GB	13.52 GB	Importance-weighted 3-bit, extra small
Q2_K	3.40	14.8 GB	48.6 GB	13.92 GB	2-bit quantization with K-quant improvements
Q3_K_S	3.50	15.2 GB	49.0 GB	14.33 GB	3-bit small quantization
IQ3_M	3.60	15.6 GB	49.4 GB	14.74 GB	Importance-weighted 3-bit, medium
Q3_K_M	3.90	16.8 GB	50.6 GB	15.97 GB	3-bit medium quantization
Q4_0	4.00	17.2 GB	51.0 GB	16.38 GB	4-bit legacy quantization
Q3_K_L	4.10	17.6 GB	51.5 GB	16.79 GB	3-bit large quantization
IQ4_XS	4.30	18.4 GB	52.3 GB	17.61 GB	Importance-weighted 4-bit, compact
Q4_1	4.50	19.3 GB	53.1 GB	18.43 GB	4-bit legacy quantization with offset
Q4_K_S	4.50	19.3 GB	53.1 GB	18.43 GB	4-bit small quantization
IQ4_NL	4.50	19.3 GB	53.1 GB	18.43 GB	Importance-weighted 4-bit, non-linear
Q4_K_M	4.80	20.5 GB	54.3 GB	19.66 GB	4-bit medium quantization — most popular sweet spot
Q4_K_L	4.90	20.9 GB	54.7 GB	20.07 GB	4-bit large quantization
Q5_K_S	5.50	23.4 GB	57.2 GB	22.53 GB	5-bit small quantization
Q5_K_M	5.70	24.2 GB	58 GB	23.34 GB	5-bit medium quantization — good quality/size tradeoff
Q5_K_L	5.80	24.6 GB	58.4 GB	23.75 GB	5-bit large quantization
Q6_K	6.60	27.9 GB	61.7 GB	27.03 GB	6-bit quantization, very good quality
Q8_0	8.00	33.6 GB	67.4 GB	32.76 GB	8-bit quantization, near-lossless

Which GPUs Can Run DeepSeek R1 Distill Qwen 32B?

Q4_K_M · 20.5 GB

Show professional

DeepSeek R1 Distill Qwen 32B (Q4_K_M) requires 20.5 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 27+ GB is recommended. Using the full 131K context window can add up to 33.8 GB, bringing total usage to 54.3 GB. 5 GPUs can run it, including NVIDIA GeForce RTX 5090, NVIDIA GeForce RTX 3090 Ti.

Runs great

— Plenty of headroom

NVIDIA GeForce RTX 5090~57 tok/s

Decent

— Enough VRAM, may be tight

NVIDIA GeForce RTX 3090 Ti~32 tok/s NVIDIA GeForce RTX 4090~32 tok/s NVIDIA GeForce RTX 3090~30 tok/s AMD Radeon RX 7900 XTX~26 tok/s

Which Devices Can Run DeepSeek R1 Distill Qwen 32B?

Q4_K_M · 20.5 GB

21 devices with unified memory can run DeepSeek R1 Distill Qwen 32B, including NVIDIA DGX H100, NVIDIA DGX A100 640GB, Mac Mini M4 Pro (24 GB).

Related Models

Derivatives (2)

DeepSeek R1 Distill Qwen 32B Abliterated DeepSeek R1 Distill Qwen 32B GGUF

Frequently Asked Questions

How much VRAM does DeepSeek R1 Distill Qwen 32B need?: DeepSeek R1 Distill Qwen 32B requires 20.5 GB of VRAM at Q4_K_M, or 33.6 GB at Q8_0. Full 131K context adds up to 33.8 GB (54.3 GB total).
VRAM = Weights + KV Cache + Overhead
Weights = 32.8B × 4.8 bits ÷ 8 = 19.7 GB
KV Cache + Overhead ≈ 0.8 GB (at 2K context + ~0.3 GB framework)
KV Cache + Overhead ≈ 34.6 GB (at full 131K context)
VRAM usage by quantization
Q4_K_M
20.5 GB
Q4_K_M + full context
54.3 GB
Learn more about VRAM estimation →
Can NVIDIA GeForce RTX 4090 run DeepSeek R1 Distill Qwen 32B?: Yes, at Q5_K_S (23.4 GB) or lower. Higher quantizations like Q5_K_M (24.2 GB) exceed the NVIDIA GeForce RTX 4090's 24 GB.
What's the best quantization for DeepSeek R1 Distill Qwen 32B?: For DeepSeek R1 Distill Qwen 32B, Q4_K_M (20.5 GB) offers the best balance of quality and VRAM usage. Q4_K_L (20.9 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 9.8 GB.
VRAM requirement by quantization
IQ2_XXS
9.8 GB~53%
Q2_K
14.8 GB~75%
IQ4_XS
18.4 GB~87%
Q4_K_M ★
20.5 GB~89%
Q4_K_L
20.9 GB~90%
Q8_0
33.6 GB~99%
★ Recommended — best balance of quality and VRAM usage.
Learn more about quantization →
Can I run DeepSeek R1 Distill Qwen 32B on a Mac?: DeepSeek R1 Distill Qwen 32B requires at least 9.8 GB at IQ2_XXS, which exceeds the unified memory of most consumer Macs. You would need a Mac Studio or Mac Pro with a high-memory configuration.
Can I run DeepSeek R1 Distill Qwen 32B locally?: Yes — DeepSeek R1 Distill Qwen 32B can run locally on consumer hardware. At Q4_K_M quantization it needs 20.5 GB of VRAM. Popular tools include Ollama, LM Studio, and llama.cpp.
How fast is DeepSeek R1 Distill Qwen 32B?: At Q4_K_M, DeepSeek R1 Distill Qwen 32B can reach ~142 tok/s on AMD Instinct MI300X. On NVIDIA GeForce RTX 4090: ~32 tok/s. Speed depends mainly on GPU memory bandwidth. Real-world results typically within ±20%.
tok/s = (bandwidth GB/s ÷ model GB) × efficiency
Example: AMD Instinct MI300X → 5300 ÷ 20.5 × 0.55 = ~142 tok/s
Estimated speed at Q4_K_M (20.5 GB)
AMD Instinct MI300X
~142 tok/s
NVIDIA GeForce RTX 4090
~32 tok/s
NVIDIA H100 SXM
~106 tok/s
AMD Instinct MI250X
~88 tok/s
Real-world results typically within ±20%. Speed depends on batch size, quantization kernel, and software stack.
Learn more about tok/s estimation →
What's the download size of DeepSeek R1 Distill Qwen 32B?: At Q4_K_M, the download is about 19.66 GB. The full-precision Q8_0 version is 32.76 GB. The smallest option (IQ2_XXS) is 9.01 GB.