All Hardware for Running LLMs Locally
101 GPUs and 176 Macs, PCs, phones and servers: which models each one runs, how fast, and at what price.
Try RTX 4090, M4 Max 128 GB or DGX Spark
Up to 32 GB, a graphics card is fastest for the money. For 64 GB and more, a Mac or a 128 GB mini PC is the cheapest way to get the memory.
- Starter GPURTX 3060 12GBabout $32912 GB · Qwen3.5 9B 37 tok/s
- Best 24 GB cardRTX 3090 / RX 7900 XTXabout $1,499
about $1,13024 GB · Qwen3.8 27B 35 / 33 tok/s - Fastest GPURTX 5090about $5,37632 GB · Qwen3.8 27B 67 tok/s
- Compact & quietMac Studio M4 Maxabout $2,49936 GB · Qwen3.8 27B 16.5 tok/s
- Most memory per dollarFramework Desktopabout $1,999128 GB · Qwen3.6 35B A3B 47 tok/s
- 128 GB with NVIDIA softwareNVIDIA DGX Sparkabout $3,999128 GB · Qwen3.6 35B A3B 54 tok/s
- Biggest modelsMac Studio M3 Ultraabout $9,499512 GB · DeepSeek V4.1 Flash 1017 40 tok/s
Browse hardware by type
Graphics cards
All 101 GPUsPopular cards, by VRAM. VRAM decides the biggest model; bandwidth decides the speed.
| GPU | VRAM | 27B tok/s | Price | |
|---|---|---|---|---|
| 32 GB | ||||
| RTX 5090 | 32 GB | 67 | about $5,376 | |
| 24 GB | ||||
| RTX 4090 | 24 GB | 38 | about $1,599 | |
| RTX 3090 | 24 GB | 35 | about $1,499 | |
| RX 7900 XTX | 24 GB | 33 | about $1,130 | |
| 16 GB | ||||
| RTX 5080 | 16 GB | 42 | about $1,690 | |
| RTX 5070 Ti | 16 GB | 39 | about $1,170 | |
| RTX 4080 SUPER | 16 GB | 32 | about $999 | |
| RTX 4070 Ti SUPER | 16 GB | 29 | about $799 | |
| RTX 4080 | 16 GB | 31 | about $1,199 | |
| RX 9070 XT | 16 GB | 26 | about $815 | |
| 12 GB | ||||
| RTX 5070 | 12 GB | 38 | about $840 | |
| RTX 4070 SUPER | 12 GB | 29 | about $599 | |
| RTX 4070 Ti | 12 GB | 29 | about $799 | |
| RTX 3060 12GB | 12 GB | 20 | about $329 | |
| 8 GB | ||||
| RTX 4060 | 8 GB | – | about $520 | |
Apple Silicon
All 148 MacsFind your chip, then your memory. Speed depends on the chip; memory decides the biggest model that fits.
Older chipsM3 MaxM3 ProM3M2 UltraM2 MaxM2 ProM2M1 UltraM1 MaxM1 ProM179 Macs
AI boxes and mini PCs
All 7 AI boxes128 GB in a small box: room for 120B mixture-of-experts models at 47–54 tok/s.
| Box | Memory | 35B MoE tok/s | Price | |
|---|---|---|---|---|
| NVIDIA GB10 and Thor · 128 GB | ||||
| NVIDIA DGX Spark | 128 GB | 54 | about $3,999 | |
| Jetson AGX Thor Developer Kit | 128 GB | 54 | about $3,499 | |
| ASUS Ascent GX10 | 128 GB | 49 | about $2,999 | |
| AMD Ryzen AI Max+ 395 · 128 GB | ||||
| Framework Desktop | 128 GB | 47 | about $1,999 | |
| GMKtec EVO-X2 | 128 GB | 47 | about $1,799 | |
| Beelink GTR9 Pro | 128 GB | 47 | about $1,985 | |
| HP Z2 Mini G1a | 128 GB | 47 | about $3,342 | |
| Minisforum MS-S1 MAX | 128 GB | 47 | about $2,299 | |
| NVIDIA Jetson · edge | ||||
| Jetson AGX Orin 64GB | 64 GB | 42 | – | |
| Jetson AGX Orin 32GB | 32 GB | 42 | – | |
| Jetson Orin NX 16GB | 16 GB | – | about $599 | |
| Jetson Orin Nano 8GB | 8 GB | – | – | |
Laptops
All 103Phones and tablets
All 8 phonesDesktops and servers
All 53 desktopsDesktops & mini PCs53
Servers & DGX systems2
AllMemory vs speed
Qwen3.6 35B A3B (35B params, 3B active) · each dot opens that hardwareMore memory runs bigger models; bandwidth sets the speed. Graphics cards are fastest but stop at 32 GB; Macs and 128 GB mini PCs trade speed for room.
Memory vs speed chart
Chart data (130 devices)
Speeds are estimates from memory bandwidth and model size at Q4. How we estimate
Frequently asked questions
How much memory do I need to run an LLM locally?
16 GB runs 9B models well. 24 GB runs 27B models, the sweet spot for most people. 64 GB runs 70B, and 128 GB runs 120B mixture-of-experts models such as GPT OSS 120B. On a Mac, count about 3.5 GB less: macOS keeps that for itself.
Should I buy a GPU or a Mac for local LLMs?
A GPU if your models fit in 24–32 GB: it is 2–4× faster for the money. A Mac or a 128 GB mini PC if you want 64 GB or more: unified memory is the cheapest way to get that much room, at lower speed.
Is the RTX 3090 still a good choice?
Yes. It has 24 GB, runs Qwen3.8 27B at about 35 tok/s, and nearly matches the speed of an RTX 4090 for LLMs. The card is out of production, so you buy it used; the RX 7900 XTX is the new 24 GB alternative.
How fast is fast enough?
About 10 tok/s reads like a fast typist. 20–30 tok/s feels instant for chat. Coding agents and long answers benefit from 50+ tok/s, because they write a lot of text per step.
Can a phone or tablet run an LLM?
Yes, small ones. Recent iPhones with 12 GB run 4B models at about 15–20 tok/s and 1B models much faster. An iPad Pro with 16 GB runs 8–9B models. Larger models need a laptop or desktop.
Why does a Mac show less usable memory than it has?
macOS and your apps keep part of the unified memory. We assume 3.5 GB, so a 24 GB Mac has about 20.5 GB for models. Phones keep more: an 8 GB iPhone has about 4.5 GB free.