All GPUs for Running LLMs Locally
Browse 101 GPUs compatible with running LLM models locally. Compare VRAM, memory bandwidth, and AI performance. Top picks and budget guide
101 GPUs
| GPU | Price |
|---|---|
| RTX 3060 8GB8 GBGDDR6 · 240 GB/s · 170 WNVIDIA · Ampere · 2022 | about $329 |
| Tesla P100 PCIe 16GBPro16 GBHBM2 · 732 GB/s · 250 WNVIDIA · Pascal · 2016 | – |
| A100 40GB PCIePro40 GBHBM2e · 1,555 GB/s · 250 WNVIDIA · Ampere · 2020 | – |
| RTX 3060 Ti8 GBGDDR6 · 448 GB/s · 200 WNVIDIA · Ampere · 2020 | about $399 |
| H200 NVLPro141 GBHBM3e · 4,800 GB/s · 600 WNVIDIA · Hopper · 2024 | – |
| Tesla M40 24GBPro24 GBGDDR5 · 288 GB/s · 250 WNVIDIA · Maxwell · 2015 | – |
| Tesla V100 PCIe 16GBPro16 GBHBM2 · 900 GB/s · 250 WNVIDIA · Volta · 2017 | – |
| RTX PRO 4000 BlackwellPro24 GBGDDR7 · 672 GB/s · 145 WNVIDIA · Blackwell · 2025 | about $1,500 |
| B300Pro288 GBHBM3e · 8,000 GB/s · 1400 WNVIDIA · Blackwell Ultra · 2025 | – |
| Instinct MI300XPro192 GBHBM3 · 5,300 GB/s · 750 WAMD · CDNA 3 · 2023 | – |
| Instinct MI325XPro256 GBHBM3e · 6,000 GB/s · 1000 WAMD · CDNA 3 · 2024 | – |
| Instinct MI350XPro288 GBHBM3e · 8,000 GB/s · 1000 WAMD · CDNA 4 · 2025 | – |
| Instinct MI355XPro288 GBHBM3e · 8,000 GB/s · 1400 WAMD · CDNA 4 · 2025 | – |
| RX 7700 XT12 GBGDDR6 · 432 GB/s · 245 WAMD · RDNA 3 · 2023 | about $449 |
| RX 90508 GBGDDR6 · 288 GB/s · 92 WAMD · RDNA 4 · 2026 | about $279 |
| Arc B57010 GBGDDR6 · 380 GB/s · 150 WIntel · Xe2 · 2025 | about $219 |
| RTX 3070 Ti8 GBGDDR6X · 608 GB/s · 290 WNVIDIA · Ampere · 2021 | about $599 |
| GH200 Grace Hopper SuperchipPro144 GBHBM3e · 4,900 GB/s · 1000 WNVIDIA · Hopper (Grace Hopper) · 2024 | – |
| L40SPro48 GBGDDR6 · 864 GB/s · 350 WNVIDIA · Ada Lovelace · 2023 | – |
| RTX PRO 4500 BlackwellPro32 GBGDDR7 · 896 GB/s · 200 WNVIDIA · Blackwell · 2025 | about $2,600 |
| RTX PRO 6000 Blackwell ServerPro96 GBGDDR7 · 1,597 GB/s · 600 WNVIDIA · Blackwell · 2025 | – |
| T4Pro16 GBGDDR6 · 320 GB/s · 70 WNVIDIA · Turing · 2018 | – |
| TITAN RTX24 GBGDDR6 · 672 GB/s · 280 WNVIDIA · Turing · 2018 | about $2,499 |
| Quadro RTX 8000Pro48 GBGDDR6 · 672 GB/s · 260 WNVIDIA · Turing · 2018 | about $9,999 |
| RTX PRO 4500 Blackwell Server EditionPro32 GBGDDR7 · 800 GB/s · 165 WNVIDIA · Blackwell · 2026 | – |
Which GPU Do You Need for AI?
The amount of VRAM is the most important specification for running LLMs locally. Most 7B parameter models require 4–8 GB of VRAM at common quantization levels, while 70B models need 24–48 GB. Memory bandwidth determines how fast the model generates tokens — faster bandwidth means faster responses.