Moonlight 16B A3B — Hardware Requirements & GPU Compatibility
Chat
Moonlight 16B A3B is a compact Mixture-of-Experts model from Moonshot AI that packs 16 billion total parameters while activating only around 3 billion per token. This efficient sparse design lets it punch well above its active parameter count, delivering surprisingly strong chat performance for its effective inference cost. The small active parameter count means Moonlight runs briskly on modest hardware, fitting comfortably on GPUs with 8–12 GB of VRAM at common quantization levels. It is an appealing choice for users who want MoE-level performance diversity without the heavy memory footprint typically associated with mixture models.
72.7K downloads
109 likesSpecifications
- Publisher
- Moonshot AI
- Family
- Moonlight
- Parameters
- 16.0B
- Architecture
- DeepseekV3ForCausalLM
- Context Length
- 8,192 tokens
- Vocabulary Size
- 163,840
- Release Date
- 2026-01-30
- License
- MIT
Get Started
HuggingFace
How Much VRAM Does Moonlight 16B A3B Need?
Select a quantization to see compatible GPUs below.
| Quantization | Bits | VRAM | + Context | File Size | Quality |
|---|---|---|---|---|---|
| BF16 | 16.00 | 32.7 GB | 34.0 GB | 31.92 GB | Brain floating point 16 — preferred for training |
Which GPUs Can Run Moonlight 16B A3B?
BF16 · 32.7 GBMoonlight 16B A3B (BF16) requires 32.7 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 43+ GB is recommended. Using the full 8K context window can add up to 1.4 GB, bringing total usage to 34.0 GB. No single GPU has enough memory — multi-GPU or cluster setups are needed.
Which Devices Can Run Moonlight 16B A3B?
BF16 · 32.7 GB13 devices with unified memory can run Moonlight 16B A3B, including NVIDIA DGX H100, NVIDIA DGX A100 640GB, Mac Studio M4 Max (36 GB).
Runs great
— Plenty of headroom
Related Models
Frequently Asked Questions
- How much VRAM does Moonlight 16B A3B need?
Moonlight 16B A3B requires 32.7 GB of VRAM at BF16. Full 8K context adds up to 1.4 GB (34.0 GB total).
VRAM = Weights + KV Cache + Overhead
Weights = 16.0B × 16 bits ÷ 8 = 31.9 GB
KV Cache + Overhead ≈ 0.8 GB (at 2K context + ~0.3 GB framework)
KV Cache + Overhead ≈ 2.1 GB (at full 8K context)
VRAM usage by quantization
BF1632.7 GBBF16 + full context34.0 GB- Can NVIDIA GeForce RTX 5090 run Moonlight 16B A3B?
No — Moonlight 16B A3B requires at least 32.7 GB at BF16, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.
- Can I run Moonlight 16B A3B on a Mac?
Moonlight 16B A3B requires at least 32.7 GB at BF16, which exceeds the unified memory of most consumer Macs. You would need a Mac Studio or Mac Pro with a high-memory configuration.
- Can I run Moonlight 16B A3B locally?
Yes — Moonlight 16B A3B can run locally on consumer hardware. At BF16 quantization it needs 32.7 GB of VRAM. Popular tools include Ollama, LM Studio, and llama.cpp.
- How fast is Moonlight 16B A3B?
At BF16, Moonlight 16B A3B can reach ~89 tok/s on AMD Instinct MI300X. Speed depends mainly on GPU memory bandwidth. Real-world results typically within ±20%.
tok/s = (bandwidth GB/s ÷ model GB) × efficiency
Example: AMD Instinct MI300X → 5300 ÷ 32.7 × 0.55 = ~89 tok/s
Estimated speed at BF16 (32.7 GB)
AMD Instinct MI300X~89 tok/sNVIDIA H100 SXM~67 tok/sAMD Instinct MI250X~55 tok/sReal-world results typically within ±20%. Speed depends on batch size, quantization kernel, and software stack.
- What's the download size of Moonlight 16B A3B?
At BF16, the download is about 31.92 GB.