llmrun
AlibabaQwen 2.5Qwen2_5_VLForConditionalGeneration

Qwen2.5 VL 72B Instruct — Hardware Requirements & GPU Compatibility

Vision

Qwen2.5-VL-72B-Instruct is Alibaba's 73-billion-parameter instruction-tuned vision-language model and represents the largest offering in the Qwen2.5-VL lineup. The card notes it can recognize text, charts, and layouts within images; act as a visual agent for controlling computers or phones; process videos exceeding an hour while pinpointing events; perform bounding-box and point localization; and generate structured outputs from scanned invoices and tables. Models of this magnitude require multi-GPU setups or machines with ample unified memory, even when quantized. The documentation states the shipped configuration supports 32,768 tokens but describes extending it via YaRN for longer inputs. Released under the Qwen license instead of Apache 2.0, you should review its terms before any commercial application. Published in January 2025, this model succeeds Qwen2-VL and reportedly improves upon that predecessor based on developer feedback.

73.4B params · dense128K contextOther96.4K downloads668 likes29.5K quant downloads

Doesn't fit a 24 GB GPU

Smallest consumer fit → Snapdragon X Elite Copilot+ PC · IQ2_XXS · 25.9 GB · ~4 tok/s

Macs from32 GB IQ2_XS · ~12 tok/s

Other

Not set up for it?

Reference GPU · RTX 4090 24 GB

Estimates, ±20% · how we estimate

Run Qwen2.5 VL 72B Instruct

One recommended way. Every other method, OS and quant is in the step-by-step guide.Each way opens a step-by-step guide for your OS and quant.

Recommended

Ollama
one command · needs Ollama installed
$ollama run hf.co/ggml-org/Qwen2.5-VL-72B-Instruct-GGUF:Q4_K_M

Q4_K_M · needs ~49.7 GB · 16K context

Download: Qwen2.5-VL-72B-Instruct-Q4_K_M.gguf47.42 GBggml-org

install, OS, other quants

Ways to runOther ways

each opens step-by-step

Run in cloud

Your own private copy · Q4_K_M · 96 GB GPU

  • The easiest way to try this model
  • Running in a few minutes, easy setup
  • Chat right here or connect any app

$5+ bonus for new users

How Much VRAM Does Qwen2.5 VL 72B Instruct Need?

Select a quantization level; the listed hardware adheres to it.

QuantizationVRAM+32KOn RTX 4090 (reference)
32.2 GB42.2 GB2-bit · K-quant — 2-bit quantization with K-quant improvementsDoes not fit: needs 36.9 GB
33.1 GB43.2 GB3-bit small — 3-bit small quantizationDoes not fit: needs 37.8 GB
36.8 GB46.8 GB3-bit medium — 3-bit medium quantizationDoes not fit: needs 41.5 GB
37.7 GB47.8 GB4-bit · legacy — 4-bit legacy quantizationDoes not fit: needs 42.4 GB
45.0 GB55.1 GB4-bit medium — 4-bit medium quantization — most popular sweet spotDoes not fit: needs 49.7 GB
53.3 GB63.4 GB5-bit medium — 5-bit medium quantization — good quality/size tradeoffDoes not fit: needs 58 GB
61.5 GB71.6 GB6-bit, very good — 6-bit quantization, very good qualityDoes not fit: needs 66.2 GB
74.4 GB84.5 GB8-bit, near-lossless — 8-bit quantization, near-losslessDoes not fit: needs 79.1 GB

Selected Q4_K_M

4-bit medium quantization

49.7GBat 16K context

Weights
44.1 GB
KV cache
5.4 GB
Overhead
0.3 GB

Doesn't fit RTX 4090 (reference) · needs 25.7 GB more

≥10 tok/s needs ≥689 GB/s · M5 Ultra ~19, M3 Ultra ~13

Or .

Which Hardware Can Run Qwen2.5 VL 72B Instruct?

Q4_K_M · 45.0 GB

Qwen2.5 VL 72B Instruct (Q4_K_M) requires 45.0 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 59+ GB is recommended. Using the full 128K context window can add up to 41.3 GB, bringing total usage to 86.3 GB. No consumer GPU has enough memory. 56 devices with unified memory can run Qwen2.5 VL 72B Instruct, including Mac Studio M5 Ultra (256 GB), Mac Studio M5 Ultra (512 GB), Mac Pro M2 Ultra (64 GB).

By VRAM tier · typical GPUs and Macs at Q4_K_M
8–16 GB · RTX 4060, RTX 3060 12GB, RTX 5060 Ti 16GBDoesn't fit — smallest quant needs 25.9 GB8 GB tier →12 GB tier →16 GB tier →
24 GB · RTX 4090referenceDoesn't fit — smallest quant needs 25.9 GB24 GB tier →
Mac 32 GB · Mac Mini M4IQ2_XS · 27.7 GB · ~4 tok/s · 16KMacs →
Mac 64 GB · Mac Studio M4 MaxQ4_K_M · 49.7 GB · ~9 tok/s · 32KMacs →

Alternatives to Qwen2.5 VL 72B Instruct

Qwen 2.5 family · 45 models →

Similar models

6
ModelScoreVRAMtok/sWhy it is similar
Llama 3.3 70B InstructMeta79.951.3 GB—needs > 24 GBsame size
NVIDIA Nemotron 3.5 Bolt 74B A9B BF16NVIDIA—54.2 GB—needs > 24 GBsame size
Kolibri 1 BF16Aleph-Alpha—22.5 GB~237same size
CodeLlama 70B Instruct HFMeta—23.2 GB~29same size
StableBeluga2Stability AI—23.5 GB~29same size
Apertus 70B 2509swiss-ai—48 GB—needs > 24 GBsame size

Specifications

Publisher
Alibaba
Family
Qwen 2.5
Parameters
73.4B
Architecture
Qwen2_5_VLForConditionalGeneration
Context Length
128,000 tokens
Vocabulary Size
152,064
Release Date
2025-01-27
License
Other

Frequently Asked Questions

How much VRAM does Qwen2.5 VL 72B Instruct need?

Qwen2.5 VL 72B Instruct requires 45.0 GB of VRAM at Q4_K_M, or 147.8 GB at BF16. Full 128K context adds up to 41.3 GB (86.3 GB total).

VRAM = Weights + KV Cache + Overhead

Weights = 73.4B × 4.8 bits ÷ 8 = 44 GB

KV Cache + Overhead ≈ 1 GB (at 2K context + ~0.3 GB framework)

Fit ratings and hardware model lists check this model with room for a 16K-token context, which needs a little more memory.

KV Cache + Overhead ≈ 42.3 GB (at full 128K context)

VRAM usage by quantization

45.0 GB
86.3 GB

Learn more about VRAM estimation →

Can NVIDIA GeForce RTX 4090 run Qwen2.5 VL 72B Instruct?

Yes, at IQ2_S (23.9 GB) or lower. Higher quantizations like IQ2_M (25.8 GB) exceed the NVIDIA GeForce RTX 4090's 24 GB.

What's the best quantization for Qwen2.5 VL 72B Instruct?

For Qwen2.5 VL 72B Instruct, Q4_K_M (45.0 GB) offers the best balance of quality and VRAM usage. Q4_K_L (45.9 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 21.2 GB.

VRAM requirement by quantization

IQ2_XXS
21.2 GB
Q2_K
32.2 GB
IQ4_XS
40.4 GB
Q4_K_M ★
45.0 GB
Q5_0
46.9 GB
BF16
147.8 GB

★ Recommended — best balance of quality and VRAM usage.

Learn more about quantization →

Can I run Qwen2.5 VL 72B Instruct on a Mac?

Yes — Mac Studio M1 Max (32 GB) and 90 other Macs can run Qwen2.5 VL 72B Instruct, the smallest at IQ2_XS. Apple Silicon uses unified memory, so the model shares RAM with the system. At Q4_K_M you need at least 45.0 GB of usable unified memory (RAM minus macOS overhead), from Mac Pro M2 Ultra (64 GB).

Can I run Qwen2.5 VL 72B Instruct locally?

Yes — Qwen2.5 VL 72B Instruct can run locally on consumer hardware. Popular tools include Ollama, LM Studio, and llama.cpp.

How fast is Qwen2.5 VL 72B Instruct?

At Q4_K_M, Qwen2.5 VL 72B Instruct can reach ~107 tok/s on AMD Instinct MI350X. Speed depends mainly on GPU memory bandwidth. Real-world results typically within ±20%.

tok/s = (bandwidth GB/s ÷ model GB) × efficiency

Example: NVIDIA B200 → 8000 ÷ 45.0 × 0.65 = ~116 tok/s

Estimated speed at Q4_K_M (45.0 GB)

~116 tok/s
~116 tok/s
~107 tok/s

Real-world results typically within ±20%. Speed depends on batch size, quantization kernel, and software stack.

Learn more about tok/s estimation →

What's the download size of Qwen2.5 VL 72B Instruct?

At Q4_K_M, the download is about 44.05 GB. The full-precision BF16 version is 146.82 GB. The smallest option (IQ2_XXS) is 20.19 GB.

Which GPUs can run Qwen2.5 VL 72B Instruct?

No single consumer GPU has enough VRAM to run Qwen2.5 VL 72B Instruct at Q4_K_M (45.0 GB). Multi-GPU or professional hardware is required.

Which devices can run Qwen2.5 VL 72B Instruct?

56 devices with unified memory can run Qwen2.5 VL 72B Instruct at Q4_K_M (45.0 GB), including ASUS Ascent GX10, Beelink GTR9 Pro (Ryzen AI Max+ 395, 128 GB), Framework Desktop (Ryzen AI Max+ 395, 128 GB), GMKtec EVO-X2 (Ryzen AI Max+ 395, 128 GB). Apple Silicon Macs use unified memory shared between CPU and GPU, making them well-suited for local LLM inference.