Orca Models — Hardware Requirements
8 Orca models from peft-internal-testing and the community, from the smallest that runs in 0.0 GB of VRAM up to 3.4B parameters. Every row links to full quantization tables and GPU compatibility.
All Orca Models by Size
| Model | Params | Runs from | Context | Publisher | Quant downloads |
|---|---|---|---|---|---|
| Orca Mini 3B | 3.4B | 1.6 GB | 2K |
Frequently Asked Questions
- How much VRAM do I need to run a Orca model?
- The smallest Orca model, Orca Mini 3B, runs from 1.6 GB of VRAM at an aggressive quantization. Larger family members need proportionally more — see the table above for every model.
- Which Orca models can I run on a 16 GB GPU?
- 1 of 1 Orca models fit in 16 GB of VRAM at some quantization, including Orca Mini 3B.
- What is the most popular Orca model to run locally?
- Orca Mini 3B is the most downloaded Orca model in local-friendly quantized formats. It runs from 1.6 GB of VRAM.