Which needs less VRAM, Dolphin 2.9.1 Llama 3 70B or Llama 3.1 405B?

At BF16, Dolphin 2.9.1 Llama 3 70B needs 142.1 GB and Llama 3.1 405B needs 891.0 GB, so Dolphin 2.9.1 Llama 3 70B is the lighter option to run locally.

What is the difference between Dolphin 2.9.1 Llama 3 70B and Llama 3.1 405B?

Dolphin 2.9.1 Llama 3 70B is a 70.6B model from dphn (Llama 3 family), while Llama 3.1 405B is a 405B model from Meta (Llama 3 family). Compare their VRAM requirements above to see which fits your GPU or Mac.

Dolphin 2.9.1 Llama 3 70B vs Llama 3.1 405B

Side-by-side comparison of VRAM requirements, quantization, context length, and hardware compatibility.

Dolphin 2.9.1 Llama 3 70B

dphn · 70.6B

Chat

Llama 3.1 405B

Meta · 405B

Chat

Specifications

	Dolphin 2.9.1 Llama 3 70B	Llama 3.1 405B
Parameters	70.6B	405B
Context	8K	—
Architecture	LlamaForCausalLM	—
License	Llama 3 Community	Llama 3.1 Community
Downloads	9.0K	514.6K
Released	Jun 2024	Sep 2024

VRAM by Quantization: Dolphin 2.9.1 Llama 3 70B vs Llama 3.1 405B

Quantization	Bits	Dolphin 2.9.1 Llama 3 70B VRAM	Llama 3.1 405B VRAM
BF16	16.00	142.1 GB	891 GB

Verdict

Dolphin 2.9.1 Llama 3 70B needs less VRAM at BF16 (142.1 GB vs 891.0 GB), so it fits on smaller GPUs. Llama 3.1 405B is the more widely downloaded of the two.

Frequently Asked Questions

Which needs less VRAM, Dolphin 2.9.1 Llama 3 70B or Llama 3.1 405B?: At BF16, Dolphin 2.9.1 Llama 3 70B needs 142.1 GB and Llama 3.1 405B needs 891.0 GB, so Dolphin 2.9.1 Llama 3 70B is the lighter option to run locally.
What is the difference between Dolphin 2.9.1 Llama 3 70B and Llama 3.1 405B?: Dolphin 2.9.1 Llama 3 70B is a 70.6B model from dphn (Llama 3 family), while Llama 3.1 405B is a 405B model from Meta (Llama 3 family). Compare their VRAM requirements above to see which fits your GPU or Mac.