Code Llama Models — Hardware Requirements
4 Code Llama models from Meta and the community, from the smallest that runs in 12.9 GB of VRAM at Q4 up to 69.0B parameters. Every row links to full quantization tables and GPU compatibility.
All Code Llama Models by Size
| Model | Params | Runs from (Q4) | Context | Publisher | Quant downloads |
|---|---|---|---|---|---|
| CodeLlama 7B HF | 6.7B | 12.9 GB | 16K | ||
| CodeLlama 7B Instruct HF | 6.7B | 12.9 GB | 16K | ||
| CodeLlama 34B Instruct HF | 33.7B | 23.8 GB | 16K | ||
| CodeLlama 34B HF | 33.7B | 23.8 GB | 16K | ||
| CodeLlama 70B Instruct HF | 69.0B | 43.0 GB | 4K |
Frequently Asked Questions
- How much VRAM do I need to run a Code Llama model?
- The smallest Code Llama model, CodeLlama 7B HF, runs from 12.9 GB of VRAM at Q4_K_M (smaller low-bit quantizations start at 3.2 GB with a noticeable quality loss). Larger family members need proportionally more — see the table above for every model.
- Which Code Llama models can I run on a 16 GB GPU?
- 2 of 5 Code Llama models fit in 16 GB of VRAM at Q4_K_M, including CodeLlama 7B HF, CodeLlama 7B Instruct HF.
- What is the most popular Code Llama model to run locally?
- CodeLlama 7B HF is the most downloaded Code Llama model in local-friendly quantized formats. It runs from 12.9 GB of VRAM at Q4_K_M.