llmrun

Best GPUs for Running LLMs Locally

101 graphics cards: which models each one runs, how fast, and at what price.

15 most viewed of 101 GPUs

GPUPrice
RTX 409024 GBGDDR6X · 1,008 GB/s · 450 WNVIDIA · Ada Lovelace · 2022about $1,599
RTX 509032 GBGDDR7 · 1,792 GB/s · 575 WNVIDIA · Blackwell · 2025about $5,376
RTX 508016 GBGDDR7 · 960 GB/s · 360 WNVIDIA · Blackwell · 2025about $1,690
RTX 5070 Ti16 GBGDDR7 · 896 GB/s · 300 WNVIDIA · Blackwell · 2025about $1,170
RTX 309024 GBGDDR6X · 936 GB/s · 350 WNVIDIA · Ampere · 2020about $1,499
RTX 4080 SUPER16 GBGDDR6X · 736 GB/s · 320 WNVIDIA · Ada Lovelace · 2024about $999
RTX 507012 GBGDDR7 · 672 GB/s · 250 WNVIDIA · Blackwell · 2025about $840
RTX 4070 Ti SUPER16 GBGDDR6X · 672 GB/s · 285 WNVIDIA · Ada Lovelace · 2024about $799
RTX 408016 GBGDDR6X · 717 GB/s · 320 WNVIDIA · Ada Lovelace · 2022about $1,199
RX 9070 XT16 GBGDDR6 · 640 GB/s · 304 WAMD · RDNA 4 · 2025about $815
RTX 4070 SUPER12 GBGDDR6X · 504 GB/s · 220 WNVIDIA · Ada Lovelace · 2024about $599
RTX 4070 Ti12 GBGDDR6X · 504 GB/s · 285 WNVIDIA · Ada Lovelace · 2023about $799
RTX 5060 Ti 16GB16 GBGDDR7 · 448 GB/s · 180 WNVIDIA · Blackwell · 2025about $429
RX 907016 GBGDDR6 · 640 GB/s · 220 WAMD · RDNA 4 · 2025about $800
RX 7900 XTX24 GBGDDR6 · 960 GB/s · 355 WAMD · RDNA 3 · 2022about $1,130
See all 101 GPUs

Best GPUs for local LLMs by budget

One row per budget, including used-market cards. The pick is the one that runs the largest model fast for that price.

Price vs speed

Higher means faster, and further left means cheaper. So the cards near the top left offer the most speed per dollar (33 popular desktop cards on Qwen3.5 9B (9B params); click a dot to see details).

$250$500$1,000$2,000$4,000050100150200tok/sRTX 4060: 28 tok/s, ≈$520, 8 GBRTX 4060 Ti 8GB: 29 tok/s, $399 MSRP, 8 GBRTX 5060: 46 tok/s, ≈$460, 8 GBRTX 5060 Ti 8GB: 46 tok/s, $379 MSRP, 8 GBRTX 3050 8GB: 23 tok/s, $249 MSRP, 8 GBRTX 3070: 46 tok/s, $499 MSRP, 8 GBRTX 3080: 78 tok/s, $699 MSRP, 10 GBRTX 2080 Ti: 63 tok/s, $999 MSRP, 11 GBRTX 5070: 69 tok/s, ≈$840, 12 GBRTX 4070 SUPER: 52 tok/s, $599 MSRP, 12 GBRTX 4070 Ti: 52 tok/s, $799 MSRP, 12 GBRX 9070 GRE: 41 tok/s, $549 MSRP, 12 GBRTX 4070: 52 tok/s, $599 MSRP, 12 GBRTX 3060 12GB: 37 tok/s, $329 MSRP, 12 GBRTX 3080 Ti: 93 tok/s, $1,199 MSRP, 12 GBRTX 5080: 98 tok/s, ≈$1,690, 16 GBRTX 5070 Ti: 92 tok/s, ≈$1,170, 16 GBRTX 4080 SUPER: 75 tok/s, $999 MSRP, 16 GBRTX 4070 Ti SUPER: 69 tok/s, $799 MSRP, 16 GBRTX 4080: 73 tok/s, $1,199 MSRP, 16 GBRX 9070 XT: 60 tok/s, ≈$815, 16 GBRTX 5060 Ti 16GB: 46 tok/s, $429 MSRP, 16 GBRX 9070: 60 tok/s, ≈$800, 16 GBRX 9060 XT 16GB: 30 tok/s, ≈$590, 16 GBRX 7800 XT: 59 tok/s, $499 MSRP, 16 GBRTX 4060 Ti 16GB: 29 tok/s, $499 MSRP, 16 GBRX 7900 GRE: 54 tok/s, $549 MSRP, 16 GBRX 7900 XT: 76 tok/s, $899 MSRP, 20 GBRTX 4090: 103 tok/s, $1,599 MSRP, 24 GBRTX 3090: 96 tok/s, $1,499 MSRP, 24 GBRX 7900 XTX: 91 tok/s, ≈$1,130, 24 GBRTX 3090 Ti: 103 tok/s, $1,999 MSRP, 24 GBRTX 5090: 183 tok/s, ≈$5,376, 32 GBRTX 5090RTX 3090RX 7900 XTXRTX 3060 12GBRTX 5070 TiRX 9070RTX 4060RTX 4090
8 GB10–12 GB16 GB20–24 GB32 GB≈ price in storesMSRP
Chart data (33 cards)
GPUVRAMPricetok/s
RTX 3050 8GB8 GBabout $24923
RTX 3060 12GB12 GBabout $32937
RTX 5060 Ti 8GB8 GBabout $37946
RTX 4060 Ti 8GB8 GBabout $39929
RTX 5060 Ti 16GB16 GBabout $42946
RTX 50608 GBabout $46046
RX 7800 XT16 GBabout $49959
RTX 4060 Ti 16GB16 GBabout $49929
RTX 30708 GBabout $49946
RTX 40608 GBabout $52028
RX 9070 GRE12 GBabout $54941
RX 7900 GRE16 GBabout $54954
RX 9060 XT 16GB16 GBabout $59030
RTX 4070 SUPER12 GBabout $59952
RTX 407012 GBabout $59952
RTX 308010 GBabout $69978
RTX 4070 Ti SUPER16 GBabout $79969
RTX 4070 Ti12 GBabout $79952
RX 907016 GBabout $80060
RX 9070 XT16 GBabout $81560
RTX 507012 GBabout $84069
RX 7900 XT20 GBabout $89976
RTX 4080 SUPER16 GBabout $99975
RTX 2080 Ti11 GBabout $99963
RX 7900 XTX24 GBabout $1,13091
RTX 5070 Ti16 GBabout $1,17092
RTX 408016 GBabout $1,19973
RTX 3080 Ti12 GBabout $1,19993
RTX 309024 GBabout $1,49996
RTX 409024 GBabout $1,599103
RTX 508016 GBabout $1,69098
RTX 3090 Ti24 GBabout $1,999103
RTX 509032 GBabout $5,376183

GPUs by VRAM

VRAM determines the largest model that fits. Open a size for each card and every model it runs.

8 GB
9B modelsQwen3.5 9B
RTX 4060, RTX 5060, RTX 3070
12 GPUs
10–12 GB
9B fast, 14B tightQwen3 14B
RTX 3060 12GB, RTX 5070, RTX 4070 SUPER
14 GPUs
16 GB
14B; 27B at 3-bitQwen3 14B, GPT OSS 20B
RTX 5070 Ti, RTX 5080, RX 9070 XT
24 GPUs
20–24 GB
27B and 35B MoEQwen3.8 27B, Qwen3.6 35B A3B
RTX 3090, RTX 4090, RX 7900 XTX
15 GPUs
32 GB
27B with long contextQwen3.8 27B
RTX 5090, Radeon AI PRO R9700, Arc Pro B70
9 GPUs
40 GB+
70B; 120B MoE from 80 GBLlama 3.3 70B Instruct, GPT OSS 120B
RTX 6000 Ada, RTX A6000, RTX PRO 6000 Blackwell
27 GPUs
Bigger than one card? Rent a 96 GB GPU from $1.33/hr →

NVIDIA, AMD or Intel

BrandGPUsRun 27BPick with 16 GB+
NVIDIA6736RTX 5070 Tiabout $1,170
AMD2511RX 7900 XTXabout $1,130
Intel94Arc A770 16GBabout $349

Frequently asked questions

How much VRAM do I need to run an LLM locally?

8 GB runs 9B models. 16 GB runs 14B models and 27B at 3-bit. 24 GB runs 27B and 35B mixture-of-experts models at 4-bit, the sweet spot for quality. 70B needs about 48 GB, so two 24 GB cards or a workstation card.

What is the best GPU for local LLMs?

For most people a 24 GB card: the RTX 3090 or the RX 7900 XTX. Both run Qwen3.8 27B near 33–35 tok/s. The RTX 5090 is the fastest single card (about 67 tok/s on the same model) and has 32 GB.

Is the RTX 3090 still worth it?

Yes. It has 24 GB and runs 27B models nearly as fast as an RTX 4090. You find it on the used market; the RX 7900 XTX is the 24 GB card available in stores if you accept AMD software.

NVIDIA or AMD for local LLMs?

NVIDIA if you want everything to work without fiddling: every tool supports CUDA first. AMD gives more VRAM for the money, and llama.cpp, Ollama and LM Studio run well on it. Intel Arc works with llama.cpp but fewer tools support it.

Does memory bandwidth or VRAM matter more?

VRAM decides which models fit; bandwidth decides how fast they run. Pick the VRAM first, then the fastest card you can afford at that size.

Can I use two GPUs together?

Yes. llama.cpp, Ollama and vLLM split a model across cards, so two 24 GB cards run a 70B model at 4-bit. Speed stays close to one card, because each token still reads the whole model once.

Related
All hardwareMacs for local LLMsHardware for codingAll modelsRent a GPU