Which needs less VRAM, Llama 3.1 70B or Llama 3.1 70B Instruct?

At Q4_K_M, Llama 3.1 70B needs 46.6 GB and Llama 3.1 70B Instruct needs 46.6 GB, so Llama 3.1 70B is the lighter option to run locally.

What is the difference between Llama 3.1 70B and Llama 3.1 70B Instruct?

Llama 3.1 70B is a 70.6B model from Meta (Llama 3 family), while Llama 3.1 70B Instruct is a 70.6B model from Meta (Llama 3 family). Compare their VRAM requirements above to see which fits your GPU or Mac.

Llama 3.1 70B vs Llama 3.1 70B Instruct

Side-by-side comparison of VRAM requirements, quantization, context length, and hardware compatibility.

Llama 3.1 70B

Meta · 70.6B

Chat

Llama 3.1 70B Instruct

Meta · 70.6B

Chat

Specifications

	Llama 3.1 70B	Llama 3.1 70B Instruct
Parameters	70.6B	70.6B
Context	—	131K
Architecture	—	—
License	Llama 3.1 Community	Llama 3.1 Community
Downloads	90.8K	709.1K
Released	Sep 2024	—

VRAM by Quantization: Llama 3.1 70B vs Llama 3.1 70B Instruct

Quantization	Bits	Llama 3.1 70B VRAM	Llama 3.1 70B Instruct VRAM
Q2_K	3.40	33.0 GB	33.0 GB
Q3_K_M	3.90	37.8 GB	37.8 GB
Q3_K_S	3.50	34.0 GB	—
Q4_K_M	4.80	46.6 GB	46.6 GB
Q5_K_M	5.70	55.3 GB	55.3 GB
Q6_K	6.60	64.0 GB	64.0 GB
Q8_0	8.00	77.6 GB	77.6 GB

Verdict

Llama 3.1 70B Instruct is the more widely downloaded of the two.

Frequently Asked Questions

Which needs less VRAM, Llama 3.1 70B or Llama 3.1 70B Instruct?: At Q4_K_M, Llama 3.1 70B needs 46.6 GB and Llama 3.1 70B Instruct needs 46.6 GB, so Llama 3.1 70B is the lighter option to run locally.
What is the difference between Llama 3.1 70B and Llama 3.1 70B Instruct?: Llama 3.1 70B is a 70.6B model from Meta (Llama 3 family), while Llama 3.1 70B Instruct is a 70.6B model from Meta (Llama 3 family). Compare their VRAM requirements above to see which fits your GPU or Mac.