ERNIE 5.5 Flash Base — Hardware Requirements & GPU Compatibility
Chat
ERNIE-5.5-Flash-Base is the pretrained base checkpoint of Baidu's ERNIE 5.5 Flash, a sparse Mixture-of-Experts language model with 280 billion parameters in total and 14 billion activated per token. It was pre-trained from scratch on 33.5 trillion tokens and released before any mid- or post-training, so it is a base model meant for fine-tuning and research, not chat. It has 43 layers, 512 routed experts (10 per token) plus one shared expert, and a hybrid of compressed and latent attention. Context length is 1,048,576 tokens for fine-tuning and inference, with training up to 131,072. It is released under the custom ERNIE 5.5 Flash Base Model License v1.0, so the terms are Baidu's own and should be checked before commercial use. FP8 and BF16 weights are published. All experts must fit in memory, so it needs a multi-GPU server even with quantization.
ERNIE 5.5 Flash Base runs on a multi-GPU server
BF16 needs ~573 GB · about 8× H100 SXM 80 GB
Other
Not set up for it?
Reference GPU · RTX 4090 24 GB
Estimates, ±20% · how we estimate
Run ERNIE 5.5 Flash Base
One recommended way. Every other method, OS and quant is in the step-by-step guide.Each way opens a step-by-step guide for your OS and quant.
Recommended
vllm serve baidu/ERNIE-5.5-Flash-Base --max-model-len 16384 --tensor-parallel-size 8Size the nodes from the table above
Download: Weights on Hugging Face
Ways to runOther ways
each opens step-by-stepRun in cloud
Rent 8× H100 SXM node · BF16
- Estimated hourly price
- ~$21.52/hr
- est., whole server
- GPUs
- 8 GPUs
- 640 GB of VRAM
$5+ bonus for new users
How Much VRAM Does ERNIE 5.5 Flash Base Need?
Select a quantization level; the listed hardware adheres to it.
| Quantization | VRAM | +32K | Runs on | |
|---|---|---|---|---|
| 572.8 GB | 573.2 GB | 16-bit · bfloat — Brain floating point 16 — preferred for training | Runs on 8× H100 SXM 80 GB |
Selected BF16
Brain floating point 16573.0GBat 16K context
- Weights
- 572.5 GB
- KV cache
- 0.2 GB
- Overhead
- 0.3 GB
Server-class: ~573 GB at 16K context · about 8× H100 SXM 80 GB
Which Hardware Can Run ERNIE 5.5 Flash Base?
BF16 · 572.8 GBERNIE 5.5 Flash Base (BF16) requires 572.8 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 745+ GB is recommended. Using the full 1049K context window can add up to 11.5 GB, bringing total usage to 584.3 GB. No consumer GPU has enough memory.
Needs server-class hardware
ERNIE 5.5 Flash Base runs on a multi-GPU server · BF16 needs ~573 GB
| Hardware | Quant | Memory | Context | Hardware page | |
|---|---|---|---|---|---|
| 8× H100 SXM 80 GB · 640 GB | BF16 | 573 GB | 256K ctx | H100 SXM → | BF16 · 573 GB · 256K ctx |
| 5× H200 SXM 141 GB · 705 GB | BF16 | 573 GB | 1M ctx | H200 SXM → | BF16 · 573 GB · 1M ctx |
| 4× B200 192 GB · 768 GB | BF16 | 573 GB | 1M ctx | B200 → | BF16 · 573 GB · 1M ctx |
Also: NVIDIA DGX A100 640GB · NVIDIA DGX H100
By VRAM tier · typical GPUs and Macs at BF16
Consumer GPUs (up to 32 GB) and Macs up to 512 GB: too small for this model (needs ≥ 573 GB)8 GB tier →12 GB tier →16 GB tier →24 GB tier →Macs →
Alternatives to ERNIE 5.5 Flash Base
ERNIE family · 7 models →Similar models
4| Model | Score | VRAM | tok/s | Why it is similar |
|---|---|---|---|---|
| Inkling SmallThinking Machines | 119.6 | 162.7 GB | —needs > 24 GB | same size |
| MiMo V2.6 Flash MOPDXiaomi | — | 186.5 GB | —needs > 24 GB | same size |
| DeepSeek V4 Flash Vision ExpDeepSeek | — | 183.3 GB | —needs > 24 GB | same size |
| Hy3Tencent | — | 182.3 GB | —needs > 24 GB | same size |
Specifications
- Publisher
- Baidu
- Family
- ERNIE
- Parameters
- 286.2B
- Active parameters
- 14.6B per token
- Architecture
- Ernie5_5_MoeForCausalLM
- Context Length
- 1,048,576 tokens
- Vocabulary Size
- 201,216
- Release Date
- 2026-10-21
- License
- Other
Frequently Asked Questions
How much VRAM does ERNIE 5.5 Flash Base need?
ERNIE 5.5 Flash Base requires 572.8 GB of VRAM at BF16. Full 1049K context adds up to 11.5 GB (584.3 GB total).
VRAM = Weights + KV Cache + Overhead
Weights = 286.2B × 16 bits ÷ 8 = 572.5 GB
KV Cache + Overhead ≈ 0.3 GB (at 2K context + ~0.3 GB framework)
Fit ratings and hardware model lists check this model with room for a 16K-token context, which needs a little more memory.
KV Cache + Overhead ≈ 11.8 GB (at full 1049K context)
VRAM usage by quantization
Can NVIDIA GeForce RTX 5090 run ERNIE 5.5 Flash Base?
No — ERNIE 5.5 Flash Base requires at least 572.8 GB at BF16, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.
Can I run ERNIE 5.5 Flash Base on a Mac?
ERNIE 5.5 Flash Base requires at least 572.8 GB at BF16. None of the 146 Macs we list has that much usable memory (the largest has 508.5 GB after macOS overhead).
Can I run ERNIE 5.5 Flash Base locally?
Yes — ERNIE 5.5 Flash Base can run locally on consumer hardware. Popular tools include Ollama, LM Studio, and llama.cpp.
What's the download size of ERNIE 5.5 Flash Base?
At BF16, the download is about 572.49 GB.
Which GPUs can run ERNIE 5.5 Flash Base?
No single consumer GPU has enough VRAM to run ERNIE 5.5 Flash Base at BF16 (572.8 GB). Multi-GPU or professional hardware is required.
Which devices can run ERNIE 5.5 Flash Base?
ERNIE 5.5 Flash Base requires at least 572.8 GB at BF16, which exceeds the usable memory of every unified-memory device we list (the largest has 508.5 GB).