Can NVIDIA GeForce RTX 5090 run GLM 5 FP8?

No — GLM 5 FP8 requires at least 211.6 GB at IQ2_XXS, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.

What's the best quantization for GLM 5 FP8?

For GLM 5 FP8, Q4_K_M (456.6 GB) offers the best balance of quality and VRAM usage. Q5_K_S (522.5 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 211.6 GB.

Can I run GLM 5 FP8 on a Mac?

GLM 5 FP8 requires at least 211.6 GB at IQ2_XXS, which exceeds the unified memory of most consumer Macs. You would need a Mac Studio or Mac Pro with a high-memory configuration.

Can I run GLM 5 FP8 locally?

Yes — GLM 5 FP8 can run locally on consumer hardware. At Q4_K_M quantization it needs 456.6 GB of VRAM. Popular tools include Ollama, LM Studio, and llama.cpp.

What's the download size of GLM 5 FP8?

At Q4_K_M, the download is about 452.35 GB. The full-precision Q8_0 version is 753.91 GB. The smallest option (IQ2_XXS) is 207.33 GB.

zai-org·GLM·GlmMoeDsaForCausalLM

GLM 5 FP8 — Hardware Requirements & GPU Compatibility

Chat

GLM 5 FP8 is the FP8 quantized release of Zhipu AI's 754 billion parameter flagship model, reducing memory requirements by storing weights in 8-bit floating point precision. This quantization roughly halves the VRAM needed compared to the full-precision version while preserving most of the model's capability across reasoning, coding, and multilingual tasks. It remains a demanding model to run locally, but FP8 quantization meaningfully lowers the hardware barrier for users with high-end multi-GPU setups.

4.3M downloads 143 likesMar 2026203K context

Specifications

Publisher: zai-org
Family: GLM
Parameters: 753.9B
Architecture: GlmMoeDsaForCausalLM
Context Length: 202,752 tokens
Vocabulary Size: 154,880
Release Date: 2026-03-11
License: MIT

Get Started

HuggingFace

zai-org/GLM-5-FP8

How Much VRAM Does GLM 5 FP8 Need?

Select a quantization to see compatible GPUs below.

Quantization	Bits	VRAM	+ Context	File Size	Quality
IQ2_XXS	2.20	211.6 GB	596.3 GB	207.33 GB	Importance-weighted 2-bit, extreme compression — significant quality loss
IQ2_M	2.70	258.7 GB	643.4 GB	254.44 GB	Importance-weighted 2-bit, medium
IQ3_XXS	3.10	296.4 GB	681.1 GB	292.14 GB	Importance-weighted 3-bit
Q2_K	3.40	324.6 GB	709.4 GB	320.41 GB	2-bit quantization with K-quant improvements
Q3_K_S	3.50	334.1 GB	718.8 GB	329.84 GB	3-bit small quantization
Q3_K_M	3.90	371.8 GB	756.5 GB	367.53 GB	3-bit medium quantization
Q4_0	4.00	381.2 GB	765.9 GB	376.96 GB	4-bit legacy quantization
IQ4_XS	4.30	409.4 GB	794.2 GB	405.23 GB	Importance-weighted 4-bit, compact
Q4_1	4.50	428.3 GB	813.0 GB	424.07 GB	4-bit legacy quantization with offset
Q4_K_S	4.50	428.3 GB	813.0 GB	424.07 GB	4-bit small quantization
IQ4_NL	4.50	428.3 GB	813.0 GB	424.07 GB	Importance-weighted 4-bit, non-linear
Q4_K_M	4.80	456.6 GB	841.3 GB	452.35 GB	4-bit medium quantization — most popular sweet spot
Q5_K_S	5.50	522.5 GB	907.3 GB	518.31 GB	5-bit small quantization
Q5_K_M	5.70	541.4 GB	926.1 GB	537.16 GB	5-bit medium quantization — good quality/size tradeoff
Q6_K	6.60	626.2 GB	1010.9 GB	621.98 GB	6-bit quantization, very good quality
Q8_0	8.00	758.1 GB	1142.9 GB	753.91 GB	8-bit quantization, near-lossless

Which GPUs Can Run GLM 5 FP8?

Q4_K_M · 456.6 GB

Show professional

GLM 5 FP8 (Q4_K_M) requires 456.6 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 594+ GB is recommended. Using the full 203K context window can add up to 384.7 GB, bringing total usage to 841.3 GB. No single GPU has enough memory — multi-GPU or cluster setups are needed.

Which Devices Can Run GLM 5 FP8?

Q4_K_M · 456.6 GB

2 devices with unified memory can run GLM 5 FP8, including NVIDIA DGX H100.

Decent

— Enough memory, may be tight

NVIDIA DGX H100~38 tok/s NVIDIA DGX A100 640GB~23 tok/s

Related Models

Frequently Asked Questions

How much VRAM does GLM 5 FP8 need?: GLM 5 FP8 requires 456.6 GB of VRAM at Q4_K_M, or 758.1 GB at Q8_0. Full 203K context adds up to 384.7 GB (841.3 GB total).
VRAM = Weights + KV Cache + Overhead
Weights = 753.9B × 4.8 bits ÷ 8 = 452.3 GB
KV Cache + Overhead ≈ 4.3 GB (at 2K context + ~0.3 GB framework)
KV Cache + Overhead ≈ 389 GB (at full 203K context)
VRAM usage by quantization
Q4_K_M
456.6 GB
Q4_K_M + full context
841.3 GB
Learn more about VRAM estimation →
Can NVIDIA GeForce RTX 5090 run GLM 5 FP8?: No — GLM 5 FP8 requires at least 211.6 GB at IQ2_XXS, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.
What's the best quantization for GLM 5 FP8?: For GLM 5 FP8, Q4_K_M (456.6 GB) offers the best balance of quality and VRAM usage. Q5_K_S (522.5 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 211.6 GB.
VRAM requirement by quantization
IQ2_XXS
211.6 GB~53%
Q3_K_S
334.1 GB~77%
Q4_1
428.3 GB~88%
Q4_K_M ★
456.6 GB~89%
Q5_K_S
522.5 GB~92%
Q8_0
758.1 GB~99%
★ Recommended — best balance of quality and VRAM usage.
Learn more about quantization →
Can I run GLM 5 FP8 on a Mac?: GLM 5 FP8 requires at least 211.6 GB at IQ2_XXS, which exceeds the unified memory of most consumer Macs. You would need a Mac Studio or Mac Pro with a high-memory configuration.
Can I run GLM 5 FP8 locally?: Yes — GLM 5 FP8 can run locally on consumer hardware. At Q4_K_M quantization it needs 456.6 GB of VRAM. Popular tools include Ollama, LM Studio, and llama.cpp.
What's the download size of GLM 5 FP8?: At Q4_K_M, the download is about 452.35 GB. The full-precision Q8_0 version is 753.91 GB. The smallest option (IQ2_XXS) is 207.33 GB.