Which needs less VRAM, Llama 2 13B Chat HF or Llama 2 7B HF?

At Q4_K_M, Llama 2 13B Chat HF needs 8.6 GB and Llama 2 7B HF needs 4.5 GB, so Llama 2 7B HF is the lighter option to run locally.

What is the difference between Llama 2 13B Chat HF and Llama 2 7B HF?

Llama 2 13B Chat HF is a 13.0B model from Meta (Llama 2 family), while Llama 2 7B HF is a 6.7B model from Meta (Llama 2 family). Compare their VRAM requirements above to see which fits your GPU or Mac.

Llama 2 13B Chat HF vs Llama 2 7B HF

Side-by-side comparison of VRAM requirements, quantization, context length, and hardware compatibility.

Llama 2 13B Chat HF

Meta · 13.0B

Chat

Llama 2 7B HF

Meta · 6.7B

Chat

Specifications

	Llama 2 13B Chat HF	Llama 2 7B HF
Parameters	13.0B	6.7B
Context	—	—
Architecture	—	—
License	Llama 2 Community	Llama 2 Community
Downloads	114.8K	778.0K
Released	—	Apr 2024

VRAM by Quantization: Llama 2 13B Chat HF vs Llama 2 7B HF

Quantization	Bits	Llama 2 13B Chat HF VRAM	Llama 2 7B HF VRAM
BF16	16.00	28.6 GB	14.8 GB
Q2_K	3.40	6.1 GB	3.1 GB
Q3_K_M	3.90	7.0 GB	3.6 GB
Q4_K_M	4.80	8.6 GB	4.5 GB
Q5_K_M	5.70	10.2 GB	5.3 GB
Q6_K	6.60	11.8 GB	6.1 GB
Q8_0	8.00	14.3 GB	7.4 GB

Verdict

Llama 2 7B HF needs less VRAM at Q4_K_M (4.5 GB vs 8.6 GB), so it fits on smaller GPUs. Llama 2 7B HF is the more widely downloaded of the two.

Frequently Asked Questions

Which needs less VRAM, Llama 2 13B Chat HF or Llama 2 7B HF?: At Q4_K_M, Llama 2 13B Chat HF needs 8.6 GB and Llama 2 7B HF needs 4.5 GB, so Llama 2 7B HF is the lighter option to run locally.
What is the difference between Llama 2 13B Chat HF and Llama 2 7B HF?: Llama 2 13B Chat HF is a 13.0B model from Meta (Llama 2 family), while Llama 2 7B HF is a 6.7B model from Meta (Llama 2 family). Compare their VRAM requirements above to see which fits your GPU or Mac.