Falcon Models — Hardware Requirements
13 Falcon models from TII UAE and the community, from the smallest that runs in 0.4 GB of VRAM at Q4 up to 41.8B parameters. Every row links to full quantization tables and GPU compatibility.
All Falcon Models by Size
| Model | Params | Runs from (Q4) | Context | Publisher | Quant downloads |
|---|---|---|---|---|---|
| Falcon Mamba Tiny Dev | 9M | 0.4 GB | — | ||
| Falcon H1 0.5B Base | 521M | 1.2 GB | 16K | ||
| Falcon 7B | 7.2B | 7.3 GB | — | ||
| Falcon 7B Instruct | 7.2B | 7.3 GB | — | ||
| Falcon Mamba 7B | 7.3B | 7.3 GB | — | ||
| Falcon H1 7B Instruct | 7.6B | 6.3 GB | 262K | ||
| Falcon 11B | 11.1B | 8.5 GB | 8K | ||
| Falcon 40B Instruct | 40B | 30.4 GB | — | ||
| Falcon 40B | 41.8B | 31.7 GB | — |
Community variants and fine-tunes(4)
| Model | Params | Runs from (Q4) | Context | Publisher | Quant downloads |
|---|---|---|---|---|---|
| Falcon H1 Tiny 90M Instruct | 91M | 0.6 GB | 262K | ||
| Falcon H1 0.5B Instruct | 521M | 1.2 GB | 16K | ||
| Falcon3 Mamba 7B Base | 7.3B | 7.3 GB | — | ||
| Falcon H1 7B Base | 7.6B | 6.3 GB | 262K |
Frequently Asked Questions
- How much VRAM do I need to run a Falcon model?
- The smallest Falcon model, Falcon Mamba Tiny Dev, runs from 0.4 GB of VRAM at Q4_K_M (smaller low-bit quantizations start at 0.0 GB with a noticeable quality loss). Larger family members need proportionally more — see the table above for every model.
- Which Falcon models can I run on a 16 GB GPU?
- 11 of 13 Falcon models fit in 16 GB of VRAM at Q4_K_M, including Falcon H1 7B Instruct, Falcon 11B, Falcon Mamba 7B.
- What is the most popular Falcon model to run locally?
- Falcon H1 7B Instruct is the most downloaded Falcon model in local-friendly quantized formats. It runs from 6.3 GB of VRAM at Q4_K_M.