All GPUs for Running LLMs Locally
Browse 101 GPUs compatible with running LLM models locally. Compare VRAM, memory bandwidth, and AI performance. Top picks and budget guide
101 GPUs
| GPU | Price |
|---|---|
| RTX PRO 4000 Blackwell SFF EditionPro24 GBGDDR7 · 432 GB/s · 70 WNVIDIA · Blackwell · 2025 | – |
| RX 6900 XT16 GBGDDR6 · 512 GB/s · 300 WAMD · RDNA 2 · 2020 | about $999 |
| RX 76008 GBGDDR6 · 288 GB/s · 165 WAMD · RDNA 3 · 2023 | about $269 |
| Arc B58012 GBGDDR6 · 456 GB/s · 190 WIntel · Battlemage · 2024 | about $249 |
| L4Pro24 GBGDDR6 · 300 GB/s · 72 WNVIDIA · Ada Lovelace · 2023 | – |
| Arc Pro B50Pro16 GBGDDR6 · 224 GB/s · 70 WIntel · Xe2 · 2025 | about $349 |
| RX 6800 XT16 GBGDDR6 · 512 GB/s · 300 WAMD · RDNA 2 · 2020 | about $649 |
| Arc A770 16GB16 GBGDDR6 · 560 GB/s · 225 WIntel · Alchemist · 2022 | about $349 |
| GTX 1080 Ti11 GBGDDR5X · 484 GB/s · 250 WNVIDIA · Pascal · 2017 | about $699 |
| L40Pro48 GBGDDR6 · 864 GB/s · 300 WNVIDIA · Ada Lovelace · 2022 | – |
| RTX 6000 AdaPro48 GBGDDR6 · 960 GB/s · 300 WNVIDIA · Ada Lovelace · 2022 | about $6,799 |
| RTX PRO 2000 BlackwellPro16 GBGDDR7 · 288 GB/s · 70 WNVIDIA · Blackwell · 2025 | – |
| V100 SXM2 32GBPro32 GBHBM2 · 900 GB/s · 300 WNVIDIA · Volta · 2018 | – |
| Instinct MI250XPro128 GBHBM2e · 3,277 GB/s · 560 WAMD · CDNA 2 · 2021 | – |
| Radeon PRO W7900Pro48 GBGDDR6 · 864 GB/s · 295 WAMD · RDNA 3 · 2023 | about $3,999 |
| RX 6700 XT12 GBGDDR6 · 384 GB/s · 230 WAMD · RDNA 2 · 2021 | about $479 |
| RX 680016 GBGDDR6 · 512 GB/s · 250 WAMD · RDNA 2 · 2020 | about $579 |
| Arc A7508 GBGDDR6 · 512 GB/s · 225 WIntel · Alchemist · 2022 | about $289 |
| A40Pro48 GBGDDR6 · 696 GB/s · 300 WNVIDIA · Ampere · 2020 | – |
| RTX 5000 Ada GenerationPro32 GBGDDR6 · 576 GB/s · 250 WNVIDIA · Ada Lovelace · 2023 | about $4,000 |
| RTX A6000Pro48 GBGDDR6 · 768 GB/s · 300 WNVIDIA · Ampere · 2020 | about $4,649 |
| Instinct MI210Pro64 GBHBM2e · 1,638 GB/s · 300 WAMD · CDNA 2 · 2022 | – |
| Radeon PRO W7800Pro32 GBGDDR6 · 576 GB/s · 260 WAMD · RDNA 3 · 2023 | about $2,499 |
| RTX 4000 Ada GenerationPro20 GBGDDR6 · 360 GB/s · 130 WNVIDIA · Ada Lovelace · 2023 | about $1,250 |
| RTX A4000Pro16 GBGDDR6 · 448 GB/s · 140 WNVIDIA · Ampere · 2021 | about $1,000 |
Which GPU Do You Need for AI?
The amount of VRAM is the most important specification for running LLMs locally. Most 7B parameter models require 4–8 GB of VRAM at common quantization levels, while 70B models need 24–48 GB. Memory bandwidth determines how fast the model generates tokens — faster bandwidth means faster responses.