Falcon 180B — Hardware Requirements & GPU Compatibility
Chat
Falcon 180B is a 180B-parameter open language model from TII UAE in the Falcon family. At BF16 it needs about 396.00 GB of VRAM — see which GPUs and Macs can run it below.
Falcon 180B runs on a 512 GB+ Mac or a multi-GPU server
Best local pick → Mac Studio (M3 Ultra, 512GB) · BF16 · 402.1 GB · ~1 tok/s
Macs from 512 GB (BF16 · ~1 tok/s)
unknown
Not set up for it?
Reference GPU · RTX 4090 24 GB
Estimates, ±20% · how we estimate
Run Falcon 180B
One recommended way. Every other method, OS and quant is in the step-by-step guide.
Recommended
vllm serve tiiuae/falcon-180B --max-model-len 16384 --tensor-parallel-size 8Size the nodes from the table above
Download: Weights on Hugging Face
Other ways
each opens step-by-stepRun in cloud
Rent 6× H100 SXM · BF16
- Estimated hourly price
- ~$16.14/hr
- est., whole server
- GPUs
- 6 GPUs
- 480 GB of VRAM
How Much VRAM Does Falcon 180B Need?
Select a quantization level; the listed hardware adheres to it.
| Quantization | VRAM | +32K | Runs on | |
|---|---|---|---|---|
| 396 GB | 409.2 GB | 16-bit · bfloat — Brain floating point 16 — preferred for training | Runs on 6× H100 SXM 80 GB |
Selected BF16
Brain floating point 16402.1GBat 16K context
- Weights
- 394.8 GB
- KV cache
- 7.0 GB
- Overhead
- 0.3 GB
Server-class: ~402.1 GB at 16K context · about 6× H100 SXM 80 GB
Which Hardware Can Run Falcon 180B?
BF16 · 396 GBFalcon 180B (BF16) requires 396 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 515+ GB is recommended. No consumer GPU has enough memory. 1 device with unified memory can run Falcon 180B, including Mac Studio (M3 Ultra, 512GB).
Needs server-class hardware
Falcon 180B runs on a 512 GB+ Mac or a multi-GPU server · BF16 needs ~402.1 GB
| Hardware | Quant | Memory | Speed | Context | Hardware page | |
|---|---|---|---|---|---|---|
| Mac 512 GB · Mac Studio (M3 Ultra, 512GB) | BF16 | 402.1 GB | ~1 tok/s | 32K ctx | Mac Studio → | BF16 · 402.1 GB · ~1 tok/s · 32K ctx |
| 6× H100 SXM 80 GB · 480 GB | BF16 | 402.1 GB | — | 32K ctx | H100 SXM → | BF16 · 402.1 GB · 32K ctx |
| 4× H200 SXM 141 GB · 564 GB | BF16 | 402.1 GB | — | 32K ctx | H200 SXM → | BF16 · 402.1 GB · 32K ctx |
| 3× B200 192 GB · 576 GB | BF16 | 402.1 GB | — | 32K ctx | B200 → | BF16 · 402.1 GB · 32K ctx |
Also: Mac Studio (M3 Ultra, 512GB) · NVIDIA DGX A100 640GB · NVIDIA DGX H100
By VRAM tier · typical GPUs and Macs at BF16
Consumer GPUs (up to 32 GB) and Macs up to 256 GB: too small for this model (needs ≥ 402.1 GB)8 GB tier →12 GB tier →16 GB tier →24 GB tier →Macs →
How good is Falcon 180B?
llmrun Score blends public benchmark boards into one number; 100 = the average open model.
Not enough data yet for an llmrun Score.
A Score needs results on enough independent boards. How the Composite Score is built
Benchmarks
The 6 boards that weigh most in the llmrun Score, of 8 with a result for this model; badges show who ran the test.
- MMLU
Epoch AI · Sep 2023Lab-reported
Lab-reported70.6%
#27/85 open
- GSM8K
Epoch AI · Sep 2023Lab-reported
Lab-reported54.4%
#29/62 open
- HellaSwag
Epoch AI · Sep 2023Lab-reported
Lab-reported89.0%
#2/47 open
- ARC Challenge
Epoch AI · Sep 2023Lab-reported
Lab-reported67.8%
#20/56 open
- WinoGrande
Epoch AI · Sep 2023Lab-reported
Lab-reported87.1%
#2/51 open
- BoolQ
Epoch AI · Sep 2023Lab-reported
Lab-reported89.0%
#2/38 open
Alternatives to Falcon 180B
Falcon family · 13 models →Similar models
2| Model | Score | VRAM | tok/s | Why it is similar |
|---|---|---|---|---|
| Qwen3.8 Flash NextAlibaba | — | 109 GB | —needs > 24 GB | same size |
| BloomBigScience | — | 122.4 GB | —needs > 24 GB | same size |
Specifications
- Publisher
- TII UAE
- Family
- Falcon
- Parameters
- 180B
- Release Date
- 2023-08-28
- License
- unknown
Frequently Asked Questions
How much VRAM does Falcon 180B need?
Falcon 180B requires 396 GB of VRAM at BF16.
VRAM = Weights + KV Cache + Overhead
Weights = 180B × 16 bits ÷ 8 = 360 GB
KV Cache + Overhead ≈ 36 GB (at 2K context + ~0.3 GB framework)
Fit ratings and hardware model lists check this model with room for a 16K-token context, which needs a little more memory.
VRAM usage by quantization
Can NVIDIA GeForce RTX 5090 run Falcon 180B?
No — Falcon 180B requires at least 396 GB at BF16, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.
Can I run Falcon 180B on a Mac?
Yes — Mac Studio (M3 Ultra, 512GB) and above can run Falcon 180B. Apple Silicon uses unified memory, so the model shares RAM with the system. At BF16 you need at least 396 GB of usable unified memory (RAM minus macOS overhead).
Can I run Falcon 180B locally?
Yes — Falcon 180B can run locally on consumer hardware. Popular tools include Ollama, LM Studio, and llama.cpp.
What's the download size of Falcon 180B?
At BF16, the download is about 360.00 GB.
Which GPUs can run Falcon 180B?
No single consumer GPU has enough VRAM to run Falcon 180B at BF16 (396 GB). Multi-GPU or professional hardware is required.
Which devices can run Falcon 180B?
1 device with unified memory can run Falcon 180B at BF16 (396 GB), including Mac Studio (M3 Ultra, 512GB). Apple Silicon Macs use unified memory shared between CPU and GPU, making them well-suited for local LLM inference.