GPU memory size · 12 cards·216 featured models fit
Best LLMs for 8 GB VRAM
Runs 9B models; 14B doesn't fit in 8 GB.
| # | Model | Speed on RTX 5060 | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 45.8 t/s | 45.8 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 89.3 t/s | 89.3 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Gemma 2 9B ITGoogle Q4_K_M · 7.3 GB · 47.7 t/s | 47.7 tok/s | Q4_K_M · 7.3 GB | |
| 4 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 103 t/s | 103 tok/s | Q4_K_M · 5.0 GB | |
| 5 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 221 t/s | 221 tok/s | Q4_K_M · 5.5 GB |
| # | Model | Speed on RTX 5060 | Reasoning Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 45.8 t/s | 45.8 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 103 t/s | 103 tok/s | Q4_K_M · 5.0 GB | |
| 3 | Qwen2.5 7B InstructAlibaba Q4_K_M · 5.8 GB · 58.4 t/s | 58.4 tok/s | Q4_K_M · 5.8 GB | |
| 4 | Gemma 2 9B ITGoogle Q4_K_M · 7.3 GB · 47.7 t/s | 47.7 tok/s | overall | Q4_K_M · 7.3 GB |
| 5 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 221 t/s | 221 tok/s | Q4_K_M · 5.5 GB |
| # | Model | Speed on RTX 5060 | Long context Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 89.3 t/s | 89.3 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 45.8 t/s | 45.8 tok/s | Q4_K_M · 6.6 GB | |
| 3 | Gemma 4 E4B ITGoogle Q4_K_M · 5.6 GB · 54.7 t/s | 54.7 tok/s | Q4_K_M · 5.6 GB | |
| 4 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 221 t/s | 221 tok/s | Q4_K_M · 5.5 GB | |
| 5 | MiniCPM5 2BOpenBMB Q4_K_M · 2.5 GB · 153 t/s | 153 tok/s | Q4_K_M · 2.5 GB |
| # | Model | Speed on RTX 5060 ↓ | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 89.3 t/s | 89.3 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 221 t/s | 221 tok/s | Q4_K_M · 5.5 GB | |
| 3 | Gemma 4 E4B ITGoogle Q4_K_M · 5.6 GB · 54.7 t/s | 54.7 tok/s | Q4_K_M · 5.6 GB | |
| 4 | Q4_K_M · 6.5 GB · 46.9 t/s | 46.9 tok/s | Q4_K_M · 6.5 GB | |
| 5 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 45.8 t/s | 45.8 tok/s | Q4_K_M · 6.6 GB |
| # | Model | Speed on RTX 5060 | Vision Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 45.8 t/s | 45.8 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 89.3 t/s | 89.3 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 103 t/s | 103 tok/s | Q4_K_M · 5.0 GB | |
| 4 | Q4_K_M · 6.5 GB · 46.9 t/s | 46.9 tok/s | Q4_K_M · 6.5 GB | |
| 5 | LFM3 Omni 2BLiquid AI Q4_K_M · 2.4 GB · 171 t/s | 171 tok/s | Q4_K_M · 2.4 GB |
GPUs with 8 GB VRAM
Compare all GPUs →Popular cards come first. Picks need a widely used card and a reasonable price.
Speed, price and value
Speed on Qwen3.5 9B (9B params) at Q4_K_M. New prices are compared with new, used with used; a card without a price for the view is shown at its MSRP. The last two columns are the speed you get per $100 of price and per 100 W of board power; higher is better.
| Card | GB/s | New price | tok/s | tok/s per $100 | tok/s per 100 W |
|---|---|---|---|---|---|
| 272 | about $520 | 27.8 | 5.3 | 24.2 | |
| 288 | about $399 | 29.4 | – | 18.4 | |
| 448 | about $460 | 45.8 | 10.0 | 31.6 | |
9 more cards with 8 GB · less common or unpriced cards, not picked | |||||
| Card | GB/s | Used price | tok/s | tok/s per $100 | tok/s per 100 W |
|---|---|---|---|---|---|
| 272 | about $335 | 27.8 | 8.3 | 24.2 | |
| 288 | about $362 | 29.4 | 8.1 | 18.4 | |
| 448 | about $388 | 45.8 | 11.8 | 31.6 | |
9 more cards with 8 GB · less common or unpriced cards, not picked | |||||
Cost per token: 8 GB against 12 GB cards
On Qwen3.5 9B (9B params, Q4_K_M, 4K context). Speed, cost per token and power per token depend on the model.
- Card, new price
- $0.64
- Power
- $0.13
Each dot is a card; hover or tap it for its numbers, click to open it. 8 GB cards without a price for the view are drawn hollow at their MSRP; other cards without one are left out. How we calculate cost per token
Next sizes down and up, plus why they're worth checking out. Speeds on Qwen3.5 9B.
Models that fit in 8 GB VRAM
Featured only216 featured models · 123 run great. Best quant that fits with a 16K context, ordered by llmrun Score (100 = average model). Fit grades how well it runs, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
Qwen3.5 9BVisionFunctions | Q4_K_M | 6.6 GB | 45.8 t/s | 262K | A74, Runs well | |
Gemma 4 E4B ITChatFunctions | Q4_K_M | 5.6 GB | 54.7 t/s | 131K | A81, Runs well | |
Ling 3.0 TinyChatFunctions | Q4_K_M | 5.5 GB | 220.7 t/s | 131K | S100, Runs great | |
MiMo V2.6 Distill Qwen 9BVisionCodeFunctions | Q4_K_M | 6.5 GB | 46.9 t/s | 262K | A77, Runs well | |
MiniCPM5 2BChatFunctions | Q4_K_M | 2.5 GB | 153.3 t/s | 131K | S97, Runs great | |
Qwen3.5 4BVisionFunctions | Q4_K_M | 3.4 GB | 89.3 t/s | 262K | S89, Runs great | |
Spark X2.6 4BChatFunctions | Q4_K_M | 4.3 GB | 98.4 t/s | 1049K | S91, Runs great | |
LFM2.5 8B A1BChatFunctions | Q4_K_M | 5.6 GB | 197.4 t/s | 128K | S100, Runs great | |
Ornith 1.5 9BChatFunctions | Q4_K_M | 6.6 GB | 45.8 t/s | 262K | A74, Runs well | |
NVIDIA Nemotron 3.5 Bolt E4B BF16ChatFunctions | Q4_K_M | 5.6 GB | 54.7 t/s | 66K | A81, Runs well | |
LFM3 1.7BChatFunctions | Q4_K_M | 2.1 GB | 202.2 t/s | 262K | S100, Runs great | |
LFM3 Omni 2BVisionChatFunctions | Q4_K_M | 2.4 GB | 171.3 t/s | 128K | S98, Runs great | |
MiniCPM V 4.7 1BVisionFunctions | Q4_K_M | 1.6 GB | 227.5 t/s | 262K | S100, Runs great | |
LFM3 VL 2BVisionChatFunctions | Q4_K_M | 2.3 GB | 187.9 t/s | 128K | S99, Runs great | |
Unlimited OCRVision | Q4_K_M | 3.3 GB | 284.6 t/s | 33K | S100, Runs great |
Tokens per second by model size. Grey rows need more than 8 GB.
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB45.8tok/s
- 14BQwen3 14B · not at 4-bitneeds 11.9 GB
- 27–31BQwen3.8 27B · not at 4-bitneeds 17.9 GB
- 35B MoEQwen3.6 35B A3B · not at 4-bitneeds 22.0 GB
- 70BLlama 3.3 70B Instruct · not at 4-bitneeds 51.3 GB
- 120B MoEGPT OSS 120B · not at 4-bitneeds 70.8 GB
Speed estimates for the RTX 5060: Q4_K_M, 4K context, single user. How we estimate
Macs keep 3.5 GB for the system, so a Mac needs 16 GB to match an 8 GB card (12.5 GB usable). They run the same models slower because memory bandwidth is lower.
| Device | Usable | Bandwidth | Speed on Qwen3.5 9B |
|---|---|---|---|
Mac Mini M2 Pro (16 GB)M2 Pro · 16 GB memory | 12.5 GB | 200 GB/s | 22.0 tok/s |
MacBook Pro 14" M1 Pro (16 GB)M1 Pro · 16 GB memory | 12.5 GB | 200 GB/s | 22.0 tok/s |
MacBook Air 13" M5 (16 GB)M5 · 16 GB memory | 12.5 GB | 153.6 GB/s | 16.9 tok/s |
iMac 24" M4 (16 GB)M4 · 16 GB memory | 12.5 GB | 120 GB/s | 13.2 tok/s |
8 GB VRAM FAQ
What models can I run with 8 GB of VRAM?
Models up to the 9B class. 123 of 216 featured models run great. The top pick is Qwen3.5 9B at Q4_K_M: it needs 6.6 GB and runs at about 45.8 tok/s on the RTX 5060. Gemma 4 E4B IT and Ling 3.0 Tiny also run well.
Is 8 GB enough for 14B models?
No. Qwen3 14B needs 11.9 GB at Q4, more than 8 GB. Rent a bigger card by the hour, or look at 12 GB VRAM.
What is the best 8 GB GPU for local LLMs?
It depends on budget. Best value new and used, and the fastest: the RTX 5060 (45.8 tok/s). Speeds are on Qwen3.5 9B.
How fast is an 8 GB card?
On the RTX 5060, Qwen3.5 9B runs at about 45.8 tok/s. Older and cheaper cards are slower: speed follows memory bandwidth, not memory size.
Is an 8 GB Mac the same as an 8 GB GPU?
No. A Mac keeps about 3.5 GB for the system, so you need a 16 GB Mac to match. It is also slower: the Mac Mini M2 Pro (16 GB) runs Qwen3.5 9B at about 22.0 tok/s.