Granite Models — Hardware Requirements
28 Granite models from IBM and the community, from the smallest that runs in 0.6 GB of VRAM at Q4 up to 32.2B parameters. Every row links to full quantization tables and GPU compatibility.
All Granite Models by Size
Community variants and fine-tunes(4)
| Model | Params | Runs from (Q4) | Context | Publisher | Quant downloads |
|---|---|---|---|---|---|
| Granite Guardian 3.2 8B Factuality Detection | 8.2B | 7.9 GB | 131K | ||
| Parable Granite 4.1 8B Claude Fable 5 | 8.4B | 8.0 GB | 131K | ||
| Granite Switch 4.1 8B Preview | 9.6B | 9.0 GB | 131K | ||
| Granite Switch 4.1 30B Preview | 32.2B | 25.1 GB | 131K |
Frequently Asked Questions
- How much VRAM do I need to run a Granite model?
- The smallest Granite model, Granite Docling 258M, runs from 0.6 GB of VRAM at Q4_K_M (smaller low-bit quantizations start at 0.5 GB with a noticeable quality loss). Larger family members need proportionally more — see the table above for every model.
- Which Granite models can I run on a 16 GB GPU?
- 29 of 34 Granite models fit in 16 GB of VRAM at Q4_K_M, including Granite 4.2 8B, Granite 4.1 8B, Granite 4.0 H Tiny.
- What is the most popular Granite model to run locally?
- Granite 4.2 8B is the most downloaded Granite model in local-friendly quantized formats. It runs from 8.3 GB of VRAM at Q4_K_M.