llmrun
AlibabaQwen 3.5Qwen3_5MoeForConditionalGeneration

Qwen3.5 122B A10B — Hardware Requirements & GPU Compatibility

VisionFunctions

Qwen3.5 122B A10B is Alibaba's 125-billion-parameter mixture-of-experts model from the Qwen 3.5 medium series, with about 10 billion parameters active per token (the A10B in its name). Because only the active experts run per token, inference is markedly faster than a dense model of comparable size, though all the weights still need to be held in memory. It is a unified vision-language model, handling images alongside text for reasoning, coding, and agentic workflows. Given its size, local inference needs a multi-GPU workstation or a large unified-memory machine; most people access a model this size through a hosted endpoint instead. It supports a 262K token context window and is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in February 2026, it sits alongside Qwen3.5-Flash, Qwen3.5-27B, and Qwen3.5-35B-A3B in Alibaba's medium-model series.

125.1B params · MoE13B active262K contextApache 2.0942.5K downloads653 likes198.3K quant downloads

Qwen3.5 122B A10B runs on a 96 GB+ Mac or a workstation GPU

Best local pick → Mac Studio (M3 Ultra, 96GB) · Q4_K_M · 75.5 GB · ~44 tok/s

Macs from48 GB IQ2_M · ~51 tok/s

Score 109.4#33 of 106 open#8 of 62 that fit ≤96 GBApache 2.0

Not set up for it?

Reference GPU · RTX 4090 24 GB

Estimates, ±20% · how we estimate

llmrun Composite Score →

109.4

Benchmarks · tier chart ↓
Among 106 open modelsHow the Composite Score is built

Run Qwen3.5 122B A10B

One recommended way. Every other method, OS and quant is in the step-by-step guide.Each way opens a step-by-step guide for your OS and quant.

Recommended

Ollama
one command · needs Ollama installed
$ollama run qwen3.5:122b

Q4_K_M · needs ~75.5 GB · 16K context

Download: Q4_K_M/Qwen3.5-122B-A10B-Q4_K_M-00001-of-00003.gguf76.54 GB + 2 partsunsloth

install, OS, other quants

Ways to runOther ways

each opens step-by-step

Run in cloud

Your own private copy · Q4_K_M · 96 GB GPU

  • The easiest way to try this model
  • Running in a few minutes, easy setup
  • Chat right here or connect any app

$5+ bonus for new users

How Much VRAM Does Qwen3.5 122B A10B Need?

Select a quantization level; the listed hardware adheres to it.

QuantizationVRAM+32KOn RTX 4090 (reference)
53.5 GB53.8 GB2-bit · K-quant — 2-bit quantization with K-quant improvementsDoes not fit: needs 53.6 GB
55.1 GB55.3 GB3-bit small — 3-bit small quantizationDoes not fit: needs 55.2 GB
61.4 GB61.6 GB3-bit medium — 3-bit medium quantizationDoes not fit: needs 61.4 GB
62.9 GB63.2 GB4-bit · legacy — 4-bit legacy quantizationDoes not fit: needs 63 GB
75.4 GB75.7 GB4-bit medium — 4-bit medium quantization — most popular sweet spotDoes not fit: needs 75.5 GB
89.5 GB89.7 GB5-bit medium — 5-bit medium quantization — good quality/size tradeoffDoes not fit: needs 89.6 GB
103.6 GB103.8 GB6-bit, very good — 6-bit quantization, very good qualityDoes not fit: needs 103.7 GB
125.5 GB125.7 GB8-bit, near-lossless — 8-bit quantization, near-losslessDoes not fit: needs 125.5 GB

Selected Q4_K_M

4-bit medium quantization

75.5GBat 16K context

Weights
75.1 GB
KV cache
0.2 GB
Overhead
0.3 GB

Doesn't fit RTX 4090 (reference) · needs 51.5 GB more

≥30 tok/s needs ≥385 GB/s · M3 Ultra ~44, M2 Ultra ~43

Or .

Which Hardware Can Run Qwen3.5 122B A10B?

Q4_K_M · 75.4 GB

Qwen3.5 122B A10B (Q4_K_M) requires 75.4 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 99+ GB is recommended. Using the full 262K context window can add up to 2.3 GB, bringing total usage to 77.8 GB. No consumer GPU has enough memory. 30 devices with unified memory can run Qwen3.5 122B A10B, including Mac Studio (M3 Ultra, 256GB), Mac Studio (M3 Ultra, 512GB), Mac Studio (M3 Ultra, 96GB).

By VRAM tier · typical GPUs and Macs at Q4_K_M
8–16 GB · RTX 4060, RTX 3060 12GB, RTX 5060 Ti 16GBDoesn't fit — smallest quant needs 34.9 GB8 GB tier →12 GB tier →16 GB tier →
24 GB · RTX 4090referenceDoesn't fit — smallest quant needs 34.9 GB24 GB tier →
Mac 32 GB · Mac Mini M4Doesn't fit — smallest quant needs 34.9 GBMacs →
Mac 64 GB · Mac Studio M4 MaxIQ3_M · 56.7 GB · ~41 tok/s · 256KMacs →

How good is Qwen3.5 122B A10B?

The llmrun Score merges 6 public benchmark boards into a single figure, where 100 represents the average for open models.

Score vs VRAM in the ≤96 GB tier

  • Qwen3.5 122B A10B
  • Other models
  • Best Score for its VRAM

#8 of 62 that fit ≤96 GB

Each dot is a model at the best quant that fits ≤96 GB. The line steps up to the best Score available at each amount of VRAM; a dot below it is beaten by a model that needs no more memory.

Full ≤96 GB ranking →

Benchmarks

The 6 boards that weigh most in the llmrun Score, of 8 with a result for this model; badges show who ran the test.

  • LMArena Text

    LMArena · Oct 2026Arena

    1416.5

    #38/188 open

    Arena
  • LMArena WebDev

    LMArena · Oct 2026Arena

    1360.4

    #37/55 open

    Arena
  • MMLU-Pro

    TIGER-LabIndependent

    86.7%

    #4/146 open

    Independent
  • SciCode

    Epoch AI · Feb 2026Lab-reported

    35.6%

    #53/72 open

    Lab-reported
  • CritPt

    Epoch AI · Feb 2026Lab-reported

    0.9%

    #41/84 open

    Lab-reported
  • Mystery Game Puzzles

    Epoch AI · Feb 2026Independent

    17.0%

    #14/20 open

    Independent
All 8 Qwen3.5 122B A10B benchmarks

In the ≤96 GB class

62 models fit ≤96 GB

Same Score, only models that fit ≤96 GB at their best quant: the ranking a ≤96 GB owner actually chooses from.

  1. #1

    MiMo V2.6 Flash RLXiaomi

    131.9

    ~153 tok/s · 93.8 GB

  2. #2

    DeepSeek V4 Flash 0731DeepSeek

    127.6

    ~103 tok/s · 95.5 GB

  3. #3

    Qwen3.8 27BAlibaba

    122.0

    ~67 tok/s · 17.9 GB

  4. #4

    Inkling SmallThinking Machines

    119.6

    ~356 tok/s · 92.9 GB

  5. #5

    DeepSeek V4 FlashDeepSeek

    115.3

    ~162 tok/s · 91.4 GB

  6. #6

    MiniMax M2.5MiniMax

    111.8

    ~139 tok/s · 93.9 GB

Full ≤96 GB ranking →RTX PRO 6000 Blackwell Workstation Edition page →

Alternatives to Qwen3.5 122B A10B

Qwen 3.5 family · 34 models →

Official models with an llmrun Score, each answering one question; numbers are at the best quant that fits a 24 GB GPU.

Similar models

10
ModelScoreVRAMtok/sWhy it is similar
MiniMax M2.5MiniMax111.8139.6 GB—needs > 24 GBsimilar score
Qwen3.6 27BAlibaba111.217.9 GB~38similar score
Qwen3 235B A22B Thinking 2507Alibaba108.4142.9 GB—needs > 24 GBsimilar score
Qwen3.6 35B A3BAlibaba106.822 GB~151similar score
Gemma 4 31B ITGoogle106.721.7 GB~32similar score
MiniMax M2.1MiniMax106.2139.6 GB—needs > 24 GBsimilar score
Gemma 4 26B A4B ITGoogle105.816.5 GB~171similar score
MiniMax M2MiniMax104.0139.6 GB—needs > 24 GBsimilar score
Qwen3 235B A22BAlibaba100.0142.9 GB—needs > 24 GBsimilar score
GLM 4.5 AirZ.ai98.667.6 GB—needs > 24 GBsimilar score
Fine-tunes & variants(4)

Specifications

Publisher
Alibaba
Family
Qwen 3.5
Parameters
125.1B
Active parameters
12.7B per token
Architecture
Qwen3_5MoeForConditionalGeneration
Context Length
262,144 tokens
Vocabulary Size
248,320
Release Date
2026-02-24
License
Apache 2.0

Resources

Frequently Asked Questions

How much VRAM does Qwen3.5 122B A10B need?

Qwen3.5 122B A10B requires 75.4 GB of VRAM at Q4_K_M, or 250.6 GB at BF16. Full 262K context adds up to 2.3 GB (77.8 GB total).

VRAM = Weights + KV Cache + Overhead

Weights = 125.1B × 4.8 bits ÷ 8 = 75.1 GB

KV Cache + Overhead ≈ 0.3 GB (at 2K context + ~0.3 GB framework)

Fit ratings and hardware model lists check this model with room for a 16K-token context, which needs a little more memory.

KV Cache + Overhead ≈ 2.7 GB (at full 262K context)

VRAM usage by quantization

75.4 GB
77.8 GB

Learn more about VRAM estimation →

Can NVIDIA GeForce RTX 5090 run Qwen3.5 122B A10B?

No — Qwen3.5 122B A10B requires at least 34.8 GB at IQ2_XXS, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.

What's the best quantization for Qwen3.5 122B A10B?

For Qwen3.5 122B A10B, Q4_K_M (75.4 GB) offers the best balance of quality and VRAM usage. Q4_K_L (77.0 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 34.8 GB.

VRAM requirement by quantization

IQ2_XXS
34.8 GB
IQ3_XS
52.0 GB
Q3_K_L
64.5 GB
Q4_K_M ★
75.4 GB
Q4_K_L
77.0 GB
BF16
250.6 GB

★ Recommended — best balance of quality and VRAM usage.

Learn more about quantization →

Can I run Qwen3.5 122B A10B on a Mac?

Yes — Mac Studio M5 Max (48 GB) and 55 other Macs can run Qwen3.5 122B A10B, the smallest at IQ2_M. Apple Silicon uses unified memory, so the model shares RAM with the system. At Q4_K_M you need at least 75.4 GB of usable unified memory (RAM minus macOS overhead), from Mac Studio (M3 Ultra, 96GB).

Can I run Qwen3.5 122B A10B locally?

Yes — Qwen3.5 122B A10B can run locally on consumer hardware. Popular tools include Ollama, LM Studio, and llama.cpp.

How fast is Qwen3.5 122B A10B?

At Q4_K_M, Qwen3.5 122B A10B can reach ~89 tok/s on AMD Instinct MI350X. Speed depends mainly on GPU memory bandwidth. Real-world results typically within ±20%.

tok/s = 1000 ÷ (active GB ÷ (bandwidth GB/s × efficiency) × 1000 + layers × routing ms)

Mixture-of-Experts: just the active experts get read for each token, along with a fixed routing cost per layer.

Example: NVIDIA B200 → 7.7 GB active ÷ (8000 × 0.65) = 1.48 ms, plus 48 layers × 0.055 ms = 2.64 ms, so 1000 ÷ 4.12 ms = ~243 tok/s

Estimated speed at Q4_K_M (75.4 GB)

~243 tok/s
~243 tok/s
~198 tok/s

Real-world results typically within ±20%. Speed depends on batch size, quantization kernel, and software stack.

Learn more about tok/s estimation →

What's the download size of Qwen3.5 122B A10B?

At Q4_K_M, the download is about 75.05 GB. The full-precision BF16 version is 250.17 GB. The smallest option (IQ2_XXS) is 34.40 GB.

Which GPUs can run Qwen3.5 122B A10B?

No single consumer GPU has enough VRAM to run Qwen3.5 122B A10B at Q4_K_M (75.4 GB). Multi-GPU or professional hardware is required.

Which devices can run Qwen3.5 122B A10B?

30 devices with unified memory can run Qwen3.5 122B A10B at Q4_K_M (75.4 GB), including ASUS Ascent GX10, Beelink GTR9 Pro (Ryzen AI Max+ 395, 128 GB), Framework Desktop (Ryzen AI Max+ 395, 128 GB), GMKtec EVO-X2 (Ryzen AI Max+ 395, 128 GB). Apple Silicon Macs use unified memory shared between CPU and GPU, making them well-suited for local LLM inference.