Qwen3 235B A22B — Hardware Requirements & GPU Compatibility
Chat
Qwen3 235B A22B is the largest model in Alibaba Cloud's Qwen 3 series, a Mixture of Experts (MoE) model with 235 billion total parameters and approximately 22 billion active parameters per forward pass. The MoE architecture enables it to deliver performance competitive with the best available open-weight models while requiring significantly less compute per token than a comparably sized dense model. It supports hybrid thinking mode for flexible chain-of-thought reasoning. Due to its massive total parameter count, running Qwen3 235B A22B locally requires substantial VRAM to load all expert weights, typically needing multiple high-end professional GPUs even at reduced precision. In heavily quantized formats it becomes accessible on workstation-class multi-GPU setups. Released under the Apache 2.0 license.
759.7K downloads
1.1K likesSpecifications
- Publisher
- Alibaba
- Family
- Qwen
- Parameters
- 235.1B
- Architecture
- Qwen3MoeForCausalLM
- Context Length
- 40,960 tokens
- Vocabulary Size
- 151,936
- Release Date
- 2025-07-26
- License
- Apache 2.0
Get Started
HuggingFace
How Much VRAM Does Qwen3 235B A22B Need?
Select a quantization to see compatible GPUs below.
| Quantization | Bits | VRAM | + Context | File Size | Quality |
|---|---|---|---|---|---|
| IQ2_XS | 2.40 | 71.0 GB | 74.8 GB | 70.53 GB | Importance-weighted 2-bit, extra small |
| IQ2_S | 2.50 | 74.0 GB | 77.7 GB | 73.47 GB | Importance-weighted 2-bit, small |
| IQ2_M | 2.70 | 79.8 GB | 83.6 GB | 79.34 GB | Importance-weighted 2-bit, medium |
| IQ3_XXS | 3.10 | 91.6 GB | 95.3 GB | 91.10 GB | Importance-weighted 3-bit |
| IQ3_XS | 3.30 | 97.5 GB | 101.2 GB | 96.98 GB | Importance-weighted 3-bit, extra small |
| Q2_K | 3.40 | 100.4 GB | 104.2 GB | 99.91 GB | 2-bit quantization with K-quant improvements |
| Q3_K_S | 3.50 | 103.3 GB | 107.1 GB | 102.85 GB | 3-bit small quantization |
| IQ3_M | 3.60 | 106.3 GB | 110.0 GB | 105.79 GB | Importance-weighted 3-bit, medium |
| Q3_K_M | 3.90 | 115.1 GB | 118.8 GB | 114.61 GB | 3-bit medium quantization |
| Q4_0 | 4.00 | 118.0 GB | 121.8 GB | 117.55 GB | 4-bit legacy quantization |
| Q3_K_L | 4.10 | 121.0 GB | 124.7 GB | 120.49 GB | 3-bit large quantization |
| IQ4_XS | 4.30 | 126.9 GB | 130.6 GB | 126.36 GB | Importance-weighted 4-bit, compact |
| IQ4_NL | 4.50 | 132.7 GB | 136.5 GB | 132.24 GB | Importance-weighted 4-bit, non-linear |
| Q4_1 | 4.50 | 132.7 GB | 136.5 GB | 132.24 GB | 4-bit legacy quantization with offset |
| Q4_K_S | 4.50 | 132.7 GB | 136.5 GB | 132.24 GB | 4-bit small quantization |
| Q4_K_M | 4.80 | 141.6 GB | 145.3 GB | 141.06 GB | 4-bit medium quantization — most popular sweet spot |
| Q4_K_L | 4.90 | 144.5 GB | 148.2 GB | 143.99 GB | 4-bit large quantization |
| Q5_K_S | 5.50 | 162.1 GB | 165.9 GB | 161.63 GB | 5-bit small quantization |
| Q5_K_M | 5.70 | 168 GB | 171.8 GB | 167.50 GB | 5-bit medium quantization — good quality/size tradeoff |
| Q6_K | 6.60 | 194.4 GB | 198.2 GB | 193.95 GB | 6-bit quantization, very good quality |
| Q8_0 | 8.00 | 235.6 GB | 239.3 GB | 235.09 GB | 8-bit quantization, near-lossless |
Which GPUs Can Run Qwen3 235B A22B?
Q4_K_M · 141.6 GBQwen3 235B A22B (Q4_K_M) requires 141.6 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 185+ GB is recommended. Using the full 41K context window can add up to 3.8 GB, bringing total usage to 145.3 GB. No single GPU has enough memory — multi-GPU or cluster setups are needed.
Which Devices Can Run Qwen3 235B A22B?
Q4_K_M · 141.6 GB4 devices with unified memory can run Qwen3 235B A22B, including NVIDIA DGX H100, NVIDIA DGX A100 640GB, Mac Pro M2 Ultra (192 GB).
Runs great
— Plenty of headroom
Decent
— Enough memory, may be tightRelated Models
Derivatives (5)
Frequently Asked Questions
- How much VRAM does Qwen3 235B A22B need?
Qwen3 235B A22B requires 141.6 GB of VRAM at Q4_K_M, or 235.6 GB at Q8_0. Full 41K context adds up to 3.8 GB (145.3 GB total).
VRAM = Weights + KV Cache + Overhead
Weights = 235.1B × 4.8 bits ÷ 8 = 141.1 GB
KV Cache + Overhead ≈ 0.5 GB (at 2K context + ~0.3 GB framework)
KV Cache + Overhead ≈ 4.2 GB (at full 41K context)
VRAM usage by quantization
Q4_K_M141.6 GBQ4_K_M + full context145.3 GB- Can NVIDIA GeForce RTX 5090 run Qwen3 235B A22B?
No — Qwen3 235B A22B requires at least 71.0 GB at IQ2_XS, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.
- What's the best quantization for Qwen3 235B A22B?
For Qwen3 235B A22B, Q4_K_M (141.6 GB) offers the best balance of quality and VRAM usage. Q4_K_L (144.5 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XS at 71.0 GB.
VRAM requirement by quantization
IQ2_XS71.0 GB~57%Q2_K100.4 GB~75%Q3_K_L121.0 GB~86%Q4_K_M ★141.6 GB~89%Q4_K_L144.5 GB~90%Q8_0235.6 GB~99%★ Recommended — best balance of quality and VRAM usage.
- Can I run Qwen3 235B A22B on a Mac?
Qwen3 235B A22B requires at least 71.0 GB at IQ2_XS, which exceeds the unified memory of most consumer Macs. You would need a Mac Studio or Mac Pro with a high-memory configuration.
- Can I run Qwen3 235B A22B locally?
Yes — Qwen3 235B A22B can run locally on consumer hardware. At Q4_K_M quantization it needs 141.6 GB of VRAM. Popular tools include Ollama, LM Studio, and llama.cpp.
- How fast is Qwen3 235B A22B?
At Q4_K_M, Qwen3 235B A22B can reach ~21 tok/s on AMD Instinct MI300X. Speed depends mainly on GPU memory bandwidth. Real-world results typically within ±20%.
tok/s = (bandwidth GB/s ÷ model GB) × efficiency
Example: AMD Instinct MI300X → 5300 ÷ 141.6 × 0.55 = ~21 tok/s
Estimated speed at Q4_K_M (141.6 GB)
AMD Instinct MI300X~21 tok/sReal-world results typically within ±20%. Speed depends on batch size, quantization kernel, and software stack.
- What's the download size of Qwen3 235B A22B?
At Q4_K_M, the download is about 141.06 GB. The full-precision Q8_0 version is 235.09 GB. The smallest option (IQ2_XS) is 70.53 GB.