llmrun
LibertAIDAIDeepSeek V4DeepseekV41ForCausalLM

DeepSeek V4.1 Flash 1017 EXL3 2.9bpw — Hardware Requirements & GPU Compatibility

ChatFunctions

DeepSeek V4.1 Flash 1017 EXL3 2.9bpw is a 105.3B-parameter open language model from LibertAIDAI in the DeepSeek V4 family. It supports a context window of up to 1,048,576 tokens. At Q4_K_M it needs about 63.52 GB of VRAM — see which GPUs and Macs can run it below.

105.3B params · dense1049K contextMIT3.3K downloads49 likes
Based on DeepSeek V4.1 Flash 1017

DeepSeek V4.1 Flash 1017 EXL3 2.9bpw runs on a 96 GB+ Mac or a workstation GPU

Best local pick → Mac Studio (M3 Ultra, 96GB) · Q4_K_M · 63.7 GB · ~9 tok/s

Macs from64 GB Q3_K_M · ~11 tok/s

MIT

Not set up for it?

Reference GPU · RTX 4090 24 GB

Estimates, ±20% · how we estimate

Run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw

One recommended way. Every other method, OS and quant is in the step-by-step guide.

Recommended

vLLM
one command · needs vLLM installed
$vllm serve LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw --max-model-len 16384

vLLM repo with its own quantisation

Download: Weights on Hugging Face

install, OS, other quants

Other ways

each opens step-by-step

Run in cloud

Fits on A100 80GB (16 GB headroom) · Q4_K_M

Generation speed
~21 tok/s
generation speed
Cost per 1M output tokens
$15.02
per 1M output tokens
Deployon Vast.aiBest optionfrom $1.13/hr (opens in new tab)
Compare GPUs →
or

How Much VRAM Does DeepSeek V4.1 Flash 1017 EXL3 2.9bpw Need?

Select a quantization level; the listed hardware adheres to it.

QuantizationVRAM+32KOn RTX 4090 (reference)
45.1 GB45.5 GB2-bit · K-quant — 2-bit quantization with K-quant improvementsDoes not fit: needs 45.3 GB
51.7 GB52.1 GB3-bit medium — 3-bit medium quantizationDoes not fit: needs 51.9 GB
63.5 GB63.9 GB4-bit medium — 4-bit medium quantization — most popular sweet spotDoes not fit: needs 63.7 GB
75.4 GB75.8 GB5-bit medium — 5-bit medium quantization — good quality/size tradeoffDoes not fit: needs 75.6 GB
87.2 GB87.6 GB6-bit, very good — 6-bit quantization, very good qualityDoes not fit: needs 87.4 GB
105.6 GB106 GB8-bit, near-lossless — 8-bit quantization, near-losslessDoes not fit: needs 105.8 GB
211.0 GB211.4 GB16-bit · bfloat — Brain floating point 16 — preferred for trainingDoes not fit: needs 211.1 GB
est.calculated, no published file yet

Selected Q4_K_M

4-bit medium quantization

63.7GBat 16K context

Weights
63.2 GB
KV cache
0.2 GB
Overhead
0.3 GB

Doesn't fit RTX 4090 (reference) · needs 39.7 GB more

Or rent A100 80GB from $1.13/hr.

Which Hardware Can Run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw?

Q4_K_M · 63.5 GB

DeepSeek V4.1 Flash 1017 EXL3 2.9bpw (Q4_K_M) requires 63.5 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 83+ GB is recommended. Using the full 1049K context window can add up to 13.4 GB, bringing total usage to 76.9 GB. No consumer GPU has enough memory. 31 devices with unified memory can run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw, including Mac Studio (M3 Ultra, 256GB), Mac Studio (M3 Ultra, 512GB), Mac Studio (M3 Ultra, 96GB).

By VRAM tier · typical GPUs and Macs at Q4_K_M
8–16 GB · RTX 4060, RTX 3060 12GB, RTX 5060 Ti 16GBDoesn't fit — smallest quant needs 45.3 GB8 GB tier →12 GB tier →16 GB tier →
24 GB · RTX 4090referenceDoesn't fit — smallest quant needs 45.3 GB24 GB tier →
Mac 32 GB · Mac Mini M4Doesn't fit — smallest quant needs 45.3 GBMacs →
Mac 64 GB · Mac Studio M4 MaxQ3_K_M · 51.9 GB · ~7 tok/s · 512KMacs →

Alternatives to DeepSeek V4.1 Flash 1017 EXL3 2.9bpw

DeepSeek V4 family · 13 models →

Similar models

6
ModelScoreVRAMtok/sWhy it is similar
Llama 4 Scout 17B 16E InstructMeta82.177 GB—needs > 24 GBsame size
C4ai Command A 03 2025Cohere79.878.7 GB—needs > 24 GBsame size
GLM 4.6VZ.ai—66 GB—needs > 24 GBsame size
Ring Flash 2.0Inclusion AI—63.1 GB—needs > 24 GBsame size
Ling Flash 2.0Inclusion AI—63.1 GB—needs > 24 GBsame size
C4ai Command R Plus 08 2024Cohere—73.8 GB—needs > 24 GBsame size

Specifications

Publisher
LibertAIDAI
Family
DeepSeek V4
Parameters
105.3B
Architecture
DeepseekV41ForCausalLM
Context Length
1,048,576 tokens
Vocabulary Size
129,280
Release Date
2026-10-17
License
MIT

Resources

Frequently Asked Questions

How much VRAM does DeepSeek V4.1 Flash 1017 EXL3 2.9bpw need?

DeepSeek V4.1 Flash 1017 EXL3 2.9bpw requires 63.5 GB of VRAM at Q4_K_M, or 211.0 GB at BF16. Full 1049K context adds up to 13.4 GB (76.9 GB total).

VRAM = Weights + KV Cache + Overhead

Weights = 105.3B × 4.8 bits ÷ 8 = 63.2 GB

KV Cache + Overhead ≈ 0.3 GB (at 2K context + ~0.3 GB framework)

Fit ratings and hardware model lists check this model with room for a 16K-token context, which needs a little more memory.

KV Cache + Overhead ≈ 13.7 GB (at full 1049K context)

VRAM usage by quantization

63.5 GB
76.9 GB

Learn more about VRAM estimation →

Can NVIDIA GeForce RTX 5090 run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw?

No — DeepSeek V4.1 Flash 1017 EXL3 2.9bpw requires at least 45.1 GB at Q2_K, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.

What's the best quantization for DeepSeek V4.1 Flash 1017 EXL3 2.9bpw?

For DeepSeek V4.1 Flash 1017 EXL3 2.9bpw, Q4_K_M (63.5 GB) offers the best balance of quality and VRAM usage. Q5_K_M (75.4 GB) provides better quality if you have the VRAM. The smallest option is Q2_K at 45.1 GB.

VRAM requirement by quantization

Q2_K
45.1 GB
Q4_K_M ★
63.5 GB
Q5_K_M
75.4 GB
Q6_K
87.2 GB
Q8_0
105.6 GB
BF16
211.0 GB

★ Recommended — best balance of quality and VRAM usage.

Learn more about quantization →

Can I run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw on a Mac?

Yes — Mac Pro M2 Ultra (64 GB) and 41 other Macs can run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw, the smallest at Q3_K_M. Apple Silicon uses unified memory, so the model shares RAM with the system. At Q4_K_M you need at least 63.5 GB of usable unified memory (RAM minus macOS overhead), from Mac Studio (M3 Ultra, 96GB).

Can I run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw locally?

Yes — DeepSeek V4.1 Flash 1017 EXL3 2.9bpw can run locally on consumer hardware. Popular tools include Ollama, LM Studio, and llama.cpp.

How fast is DeepSeek V4.1 Flash 1017 EXL3 2.9bpw?

At Q4_K_M, DeepSeek V4.1 Flash 1017 EXL3 2.9bpw can reach ~76 tok/s on AMD Instinct MI350X. Speed depends mainly on GPU memory bandwidth. Real-world results typically within ±20%.

tok/s = (bandwidth GB/s ÷ model GB) × efficiency

Example: NVIDIA B200 → 8000 ÷ 63.5 × 0.65 = ~82 tok/s

Estimated speed at Q4_K_M (63.5 GB)

~82 tok/s
~82 tok/s
~76 tok/s

Real-world results typically within ±20%. Speed depends on batch size, quantization kernel, and software stack.

Learn more about tok/s estimation →

What's the download size of DeepSeek V4.1 Flash 1017 EXL3 2.9bpw?

At Q4_K_M, the download is about 63.19 GB. The full-precision BF16 version is 210.64 GB. The smallest option (Q2_K) is 44.76 GB.

Which GPUs can run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw?

No single consumer GPU has enough VRAM to run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw at Q4_K_M (63.5 GB). Multi-GPU or professional hardware is required.

Which devices can run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw?

31 devices with unified memory can run DeepSeek V4.1 Flash 1017 EXL3 2.9bpw at Q4_K_M (63.5 GB), including ASUS Ascent GX10, Beelink GTR9 Pro (Ryzen AI Max+ 395, 128 GB), Framework Desktop (Ryzen AI Max+ 395, 128 GB), GMKtec EVO-X2 (Ryzen AI Max+ 395, 128 GB). Apple Silicon Macs use unified memory shared between CPU and GPU, making them well-suited for local LLM inference.