GPU memory size · 10 cards·276 featured models fit
Best LLMs for 12 GB VRAM
Runs 9B models; 14B only just fits in 12 GB.
| # | Model | Speed on RTX 5070 | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 68.7 t/s | 68.7 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 134 t/s | 134 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Gemma 3 12B ITGoogle Q4_K_M · 10.9 GB · 54.3 t/s | 54.3 tok/s | Q4_K_M · 10.9 GB | |
| 4 | Gemma 2 9B ITGoogle Q4_K_M · 7.3 GB · 71.6 t/s | 71.6 tok/s | Q4_K_M · 7.3 GB | |
| 5 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 289 t/s | 289 tok/s | Q4_K_M · 5.5 GB |
| # | Model | Speed on RTX 5070 | Coding Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 7.9 GB · 82.4 t/s | 82.4 tok/s | Q4_K_M · 7.9 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 68.7 t/s | 68.7 tok/s | overall | Q4_K_M · 6.6 GB |
| 3 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 134 t/s | 134 tok/s | overall | Q4_K_M · 3.4 GB |
| 4 | Gemma 4 12B ITGoogle Q4_K_M · 9.0 GB · 53.1 t/s | 53.1 tok/s | Q4_K_M · 9.0 GB | |
| 5 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 289 t/s | 289 tok/s | Q4_K_M · 5.5 GB |
| # | Model | Speed on RTX 5070 | Reasoning Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 68.7 t/s | 68.7 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3 8BAlibaba Q4_K_M · 7.6 GB · 79.1 t/s | 79.1 tok/s | Q4_K_M · 7.6 GB | |
| 3 | Q4_K_M · 7.9 GB · 82.4 t/s | 82.4 tok/s | Q4_K_M · 7.9 GB | |
| 4 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 154 t/s | 154 tok/s | Q4_K_M · 5.0 GB | |
| 5 | Gemma 4 12B ITGoogle Q4_K_M · 9.0 GB · 53.1 t/s | 53.1 tok/s | Q4_K_M · 9.0 GB |
| # | Model | Speed on RTX 5070 | Long context Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 68.7 t/s | 68.7 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 134 t/s | 134 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Q4_K_M · 7.9 GB · 82.4 t/s | 82.4 tok/s | Q4_K_M · 7.9 GB | |
| 4 | Mistral Nemo Instruct 2407Mistral AI Q4_K_M · 11.0 GB · 54.1 t/s | 54.1 tok/s | Q4_K_M · 11.0 GB | |
| 5 | Gemma 4 12B ITGoogle Q4_K_M · 9.0 GB · 53.1 t/s | 53.1 tok/s | Q4_K_M · 9.0 GB |
| # | Model | Speed on RTX 5070 ↓ | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 134 t/s | 134 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 289 t/s | 289 tok/s | Q4_K_M · 5.5 GB | |
| 3 | Qwen3 8BAlibaba Q4_K_M · 7.6 GB · 79.1 t/s | 79.1 tok/s | Q4_K_M · 7.6 GB | |
| 4 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 68.7 t/s | 68.7 tok/s | Q4_K_M · 6.6 GB | |
| 5 | Gemma 4 12B ITGoogle Q4_K_M · 9.0 GB · 53.1 t/s | 53.1 tok/s | Q4_K_M · 9.0 GB |
| # | Model | Speed on RTX 5070 | Vision Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 68.7 t/s | 68.7 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 134 t/s | 134 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Gemma 3 12B ITGoogle Q4_K_M · 10.9 GB · 54.3 t/s | 54.3 tok/s | Q4_K_M · 10.9 GB | |
| 4 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 154 t/s | 154 tok/s | Q4_K_M · 5.0 GB | |
| 5 | Q4_K_M · 6.5 GB · 70.3 t/s | 70.3 tok/s | Q4_K_M · 6.5 GB |
GPUs with 12 GB VRAM
Compare all GPUs →Popular cards come first. Picks need a widely used card and a reasonable price.
Speed, price and value
Speed on Qwen3.5 9B (9B params) at Q4_K_M. New prices are compared with new, used with used; a card without a price for the view is shown at its MSRP. The last two columns are the speed you get per $100 of price and per 100 W of board power; higher is better.
| Card | GB/s | New price | tok/s | tok/s per $100 | tok/s per 100 W |
|---|---|---|---|---|---|
| 672 | about $840 | 68.7 | 8.2 | 27.5 | |
| 504 | about $599 | 51.5 | – | 23.4 | |
| 504 | about $799 | 51.5 | – | 18.1 | |
| 504 | about $599 | 51.5 | – | 25.8 | |
| 432 | about $549 | 40.8 | – | 18.5 | |
| 360 | about $329 | 36.8 | – | 21.6 | |
4 more cards with 12 GB · less common or unpriced cards, not picked | |||||
| Card | GB/s | Used price | tok/s | tok/s per $100 | tok/s per 100 W |
|---|---|---|---|---|---|
| 672 | about $736 | 68.7 | 9.3 | 27.5 | |
| 504 | about $675 | 51.5 | 7.6 | 23.4 | |
| 504 | about $722 | 51.5 | 7.1 | 18.1 | |
| 504 | about $575 | 51.5 | 9.0 | 25.8 | |
| 432 | about $549 | 40.8 | – | 18.5 | |
| 360 | about $329 | 36.8 | 11.2 | 21.6 | |
4 more cards with 12 GB · less common or unpriced cards, not picked | |||||
Cost per token: 12 GB against 8 and 16 GB cards
On Qwen3.5 9B (9B params, Q4_K_M, 4K context). Speed, cost per token and power per token depend on the model.
- Card, used price
- $0.57
- Power
- $0.19
Renting costs as much as buying after ≈5,000 hours (≈3.4 years at 4 h a day).
Each dot is a card; hover or tap it for its numbers, click to open it. 12 GB cards without a price for the view are drawn hollow at their MSRP; other cards without one are left out. How we calculate cost per token
Next sizes down and up, plus why they're worth checking out. Speeds on Qwen3.5 9B.
Models that fit in 12 GB VRAM
Featured only276 featured models · 200 run great. Best quant that fits with a 16K context, ordered by llmrun Score (100 = average model). Fit grades how well it runs, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
| Q4_K_M | 7.3 GB | 43.4 t/s | 131K | A77, Runs well | ||
Qianfan OCRVision | Q4_K_M | 4.7 GB | 70.3 t/s | 33K | S86, Runs great | |
Nanonets OCR2 3BVision | Q4_K_M | 3.2 GB | 89.0 t/s | 128K | S89, Runs great | |
Llava 1.5 7B HFVision | Q4_K_M | 5.0 GB | 50.2 t/s | 4K | A79, Runs well | |
Qwen3 VL 2B InstructVisionFunctions | Q4_K_M | 3.5 GB | 129.3 t/s | 262K | S95, Runs great | |
Idefics3 8B Llama3Vision | Q4_K_M | 7.5 GB | 41.4 t/s | 131K | A76, Runs well | |
| Q4_K_M | 7.8 GB | 34.5 t/s | 8K | A73, Runs well | ||
Llama 3.2 11B Vision InstructVisionFunctions | Q4_K_M | 9.8 GB | 33.2 t/s | — | A70, Runs well | |
| Q4_K_M | 6.8 GB | 47.6 t/s | 33K | A79, Runs well | ||
GLM 4.7 FlashChatFunctions | IQ2_M | 11.7 GB | 100.8 t/s | 203K | C44, Usable | |
Qwen2.5 Omni 3BChat | Q4_K_M | 6.0 GB | 64.1 t/s | — | A84, Runs well | |
Eurus 2 7B PRIMEChat | Q4_K_M | 5.1 GB | 46.9 t/s | 4K | A78, Runs well | |
Llama 3.2 1B InstructChatFunctions | Q4_K_M | 2.3 GB | 285.4 t/s | 131K | S100, Runs great | |
Dots.ocrVision | Q4_K_M | 2.6 GB | 107.3 t/s | 131K | S92, Runs great | |
Translategemma 4B ITVision | Q4_K_M | 5 GB | 82.4 t/s | — | S88, Runs great |
Tokens per second by model size. Grey rows need more than 12 GB.
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB68.7tok/s
- 46.0tok/s
- 27–31BQwen3.8 27B · not at 4-bitneeds 17.9 GB
- 35B MoEQwen3.6 35B A3B · not at 4-bitneeds 22.0 GB
- 70BLlama 3.3 70B Instruct · not at 4-bitneeds 51.3 GB
- 120B MoEGPT OSS 120B · not at 4-bitneeds 70.8 GB
Speed estimates for the RTX 5070: Q4_K_M, 4K context, single user. How we estimate
Macs keep 3.5 GB for the system, so a Mac needs 16 GB to match a 12 GB card (12.5 GB usable). They run the same models slower because memory bandwidth is lower.
| Device | Usable | Bandwidth | Speed on Qwen3.5 9B |
|---|---|---|---|
Mac Mini M2 Pro (16 GB)M2 Pro · 16 GB memory | 12.5 GB | 200 GB/s | 22.0 tok/s |
MacBook Pro 14" M1 Pro (16 GB)M1 Pro · 16 GB memory | 12.5 GB | 200 GB/s | 22.0 tok/s |
MacBook Air 13" M5 (16 GB)M5 · 16 GB memory | 12.5 GB | 153.6 GB/s | 16.9 tok/s |
iMac 24" M4 (16 GB)M4 · 16 GB memory | 12.5 GB | 120 GB/s | 13.2 tok/s |
12 GB VRAM FAQ
What models can I run with 12 GB of VRAM?
Models up to the 14B class. 200 of 276 featured models run great. The top pick is Qwen3.5 9B at Q4_K_M: it needs 6.6 GB and runs at about 68.7 tok/s on the RTX 5070. Gemma 4 12B IT and Ling 3.0 Tiny also run well.
Is 12 GB enough for 27–31B models?
No. Qwen3.8 27B needs 17.9 GB at Q4, more than 12 GB. Rent a bigger card by the hour, or look at 24 GB VRAM.
What is the best 12 GB GPU for local LLMs?
It depends on budget. Best value new and fastest: the RTX 5070 (68.7 tok/s); Best value on the used market: the RTX 3060 12GB (36.8 tok/s). Speeds are on Qwen3.5 9B.
How fast is a 12 GB card?
On the RTX 5070, Qwen3.5 9B runs at about 68.7 tok/s and Qwen3 14B at about 46.0 tok/s. Older and cheaper cards are slower: speed follows memory bandwidth, not memory size.
Is renting a 12 GB GPU cheaper than buying one?
An RTX 5070 rents from $0.17/hr. Renting costs as much as buying after ≈5,000 hours (≈3.4 years at 4 h a day).
Is a 12 GB Mac the same as a 12 GB GPU?
No. A Mac keeps about 3.5 GB for the system, so you need a 16 GB Mac to match. It is also slower: the Mac Mini M2 Pro (16 GB) runs Qwen3.5 9B at about 22.0 tok/s.