All GPUs for Running LLMs Locally
Browse 101 GPUs compatible with running LLM models locally. Compare VRAM, memory bandwidth, and AI performance. Top picks and budget guide
101 GPUs
| GPU | Price |
|---|---|
| RTX 4060 Ti 8GB8 GBGDDR6 · 288 GB/s · 160 WNVIDIA · Ada Lovelace · 2023 | about $399 |
| RTX 50608 GBGDDR7 · 448 GB/s · 145 WNVIDIA · Blackwell · 2025 | about $460 |
| H100 SXMPro80 GBHBM3 · 3,352 GB/s · 700 WNVIDIA · Hopper · 2022 | – |
| Tesla P40Pro24 GBGDDR5 · 346 GB/s · 250 WNVIDIA · Pascal · 2016 | – |
| Arc Pro B60Pro24 GBGDDR6 · 456 GB/s · 200 WIntel · Xe2 · 2025 | about $599 |
| Arc Pro B70Pro32 GBGDDR6 · 608 GB/s · 230 WIntel · Xe2 · 2026 | about $949 |
| RTX 2080 Ti11 GBGDDR6 · 616 GB/s · 250 WNVIDIA · Turing · 2018 | about $999 |
| RX 7800 XT16 GBGDDR6 · 624 GB/s · 263 WAMD · RDNA 3 · 2023 | about $499 |
| RTX 308010 GBGDDR6X · 760 GB/s · 320 WNVIDIA · Ampere · 2020 | about $699 |
| RTX 4060 Ti 16GB16 GBGDDR6 · 288 GB/s · 165 WNVIDIA · Ada Lovelace · 2023 | about $499 |
| H100 PCIePro80 GBHBM2e · 2,039 GB/s · 350 WNVIDIA · Hopper · 2022 | – |
| RX 7900 GRE16 GBGDDR6 · 576 GB/s · 260 WAMD · RDNA 3 · 2024 | about $549 |
| Arc Pro B60 Dual 48GBPro48 GBGDDR6 · 456 GB/s · 400 WIntel · Xe2 · 2025 | about $1,200 |
| RTX 5090 Laptop GPULaptop24 GBGDDR7 · 896 GB/s · 150 WNVIDIA · Blackwell · 2025 | – |
| Arc Pro B65Pro32 GBGDDR6 · 608 GB/s · 200 WIntel · Xe2 · 2026 | about $910 |
| RTX 5060 Ti 8GB8 GBGDDR7 · 448 GB/s · 180 WNVIDIA · Blackwell · 2025 | about $379 |
| RX 7900 XT20 GBGDDR6 · 800 GB/s · 315 WAMD · RDNA 3 · 2022 | about $899 |
| A100 80GB SXMPro80 GBHBM2e · 2,039 GB/s · 400 WNVIDIA · Ampere · 2020 | – |
| RTX 3050 8GB8 GBGDDR6 · 224 GB/s · 130 WNVIDIA · Ampere · 2022 | about $249 |
| RTX 30708 GBGDDR6 · 448 GB/s · 220 WNVIDIA · Ampere · 2020 | about $499 |
| RTX 3080 Ti12 GBGDDR6X · 912 GB/s · 350 WNVIDIA · Ampere · 2021 | about $1,199 |
| RTX 4090 Laptop GPULaptop16 GBGDDR6 · 576 GB/s · 150 WNVIDIA · Ada Lovelace · 2023 | – |
| RTX PRO 5000 BlackwellPro72 GBGDDR7 · 1,344 GB/s · 300 WNVIDIA · Blackwell · 2025 | about $4,500 |
| RTX PRO 6000 Blackwell Max-QPro96 GBGDDR7 · 1,792 GB/s · 300 WNVIDIA · Blackwell · 2025 | about $8,565 |
| RX 7600 XT16 GBGDDR6 · 288 GB/s · 190 WAMD · RDNA 3 · 2024 | about $329 |
Which GPU Do You Need for AI?
The amount of VRAM is the most important specification for running LLMs locally. Most 7B parameter models require 4–8 GB of VRAM at common quantization levels, while 70B models need 24–48 GB. Memory bandwidth determines how fast the model generates tokens — faster bandwidth means faster responses.