GPUs with 40 GB or more VRAM for Running LLMs
Browse 27 GPUs with 40 GB or more VRAM compatible with running LLM models locally. Compare VRAM, memory bandwidth, and AI performance. Top picks and budget guide
27 GPUsClear filters
| GPU | Price |
|---|---|
| RTX A6000Pro48 GBGDDR6 · 768 GB/s · 300 WNVIDIA · Ampere · 2020 | about $4,649 |
| Instinct MI210Pro64 GBHBM2e · 1,638 GB/s · 300 WAMD · CDNA 2 · 2022 | – |
Which GPU Do You Need for AI?
The amount of VRAM is the most important specification for running LLMs locally. Most 7B parameter models require 4–8 GB of VRAM at common quantization levels, while 70B models need 24–48 GB. Memory bandwidth determines how fast the model generates tokens — faster bandwidth means faster responses.