llmrun

Best LLMs for 12 GB VRAM

Mid-range (RTX 3060, RTX 4070, RTX 5070) — 7-13B models at Q4-Q6

← 8 GB VRAM16 GB VRAM →

12 GB is the sweet spot for entry into local AI. It runs 7B–13B models at good quality quantizations, making it a practical and affordable starting point for running LLMs on your own hardware.

This memory tier, common on GPUs like the RTX 3060 12GB, is surprisingly capable for local AI. You can run Llama 3 8B, Mistral 7B, and similar 7B models at Q4_K_M quantization with decent token generation speed. Smaller models like Phi 3 Mini (3.8B) run at Q6 or Q8 with room to spare. Reaching up to 13B models is possible at Q2–Q3 quantization, though quality trade-offs become more noticeable.

Runs Well

  • 7B models at Q4_K_M quality
  • Small models (3B–4B) at Q5–Q8
  • Chat and coding assistants for everyday use

Challenging

  • 13B models only at Q2–Q3 (lower quality)
  • 14B+ models do not fit
  • Context windows limited for 7B+ models

GPUs with ~12.0 GB VRAM

All 13 GPUs

Intel Arc B570

Intel · Xe2 (Battlemage), BMG-G21

10 GB
380.0 GB/s2,304 SP150W TDP$219

NVIDIA GeForce RTX 3080

NVIDIA · Ampere

10 GB
760.3 GB/s8,704 CUDA320W TDP$699

NVIDIA GeForce RTX 2080 Ti

NVIDIA · Turing

11 GB
616.0 GB/s4,352 CUDA250W TDP$999

NVIDIA GeForce GTX 1080 Ti

NVIDIA · Pascal

11 GB
484.4 GB/s3,584 CUDA250W TDP$699

AMD Radeon RX 7700 XT

AMD · RDNA 3

12 GB
432.0 GB/s3,456 SP245W TDP$449

NVIDIA GeForce RTX 4070 Ti

NVIDIA · Ada Lovelace

12 GB
504.0 GB/s7,680 CUDA285W TDP$799

Models That Fit in 12 GB VRAM

Speed estimated for NVIDIA GeForce RTX 3080 Ti

80 models · 9 excellent · 10 good

LLM models ranked by compatibility and performance
ModelVRAMGrade
Phi 3 Mini 4k Instruct3.8B
Q4_K_M·174.4 t/s tok/s·4K ctx·EASY RUN
3.4 GBC43
Yi 6B6.1B
Q4_K_M·145.7 t/s tok/s·4K ctx·FAIR FIT
4.1 GBB49
Qwen3 4B Instruct 25074.0B
Q4_K_M·204.5 t/s tok/s·262K ctx·EASY RUN
2.9 GBC39
Gemma 3 4B IT4.3B
Q4_K_M·208.8 t/s tok/s·EASY RUN
2.8 GBC39
Phi 4 Mini Instruct3.8B
Q4_K_M·206.6 t/s tok/s·131K ctx·EASY RUN
2.9 GBC39
Phi 22.8B
Q4_K_M·224.6 t/s tok/s·2K ctx·EASY RUN
2.6 GBC37
Phi 4 Mini Reasoning3.8B
Q4_K_M·206.6 t/s tok/s·131K ctx·EASY RUN
2.9 GBC39
Llama 3.2 3B Instruct3.2B
Q4_K_M·279.7 t/s tok/s·131K ctx·EASY RUN
2.1 GBC34
Qwen2.5 Coder 3B3.1B
Q4_K_M·265.9 t/s tok/s·33K ctx·EASY RUN
2.2 GBC35
SmolLM3 3B3.1B
Q4_K_M·257.9 t/s tok/s·66K ctx·EASY RUN
2.3 GBC35
Granite 4.0 Micro3.4B
Q4_K_M·236.3 t/s tok/s·131K ctx·EASY RUN
2.5 GBC36
Llama 3.2 1B Instruct1.2B
Q4_K_M·723.2 t/s tok/s·131K ctx·EASY RUN
0.8 GBD29
Mistral Small 24B Instruct 250123.6B
IQ3_M·52.3 t/s tok/s·33K ctx·POOR FIT
11.3 GBC36
Qwen3.6 27B27.8B
IQ3_XXS·51.5 t/s tok/s·262K ctx·POOR FIT
11.5 GBD29
Magistral Small 250623.6B
IQ3_M·52.3 t/s tok/s·41K ctx·POOR FIT
11.3 GBC36
Starcoder2 3B3.0B
Q4_K_M·272.0 t/s tok/s·16K ctx·EASY RUN
2.2 GBC34

Frequently Asked Questions

What models can I run with 12.0 GB VRAM?

With 12.0 GB VRAM, you can run 1059 LLM models at various quantization levels. Popular models that fit well include Gemma 3 12B IT, Gemma 4 12B IT, Llama 2 13B Chat HF. 44 models achieve excellent performance at this VRAM level. This is the most popular entry point for local AI. Most 7B models — the workhorse size for chat and coding — fit comfortably.

Is 12.0 GB enough for local AI?

12.0 GB is a practical entry point for local AI. You can run 1059 models, including popular choices like Llama 3 8B and Mistral 7B at good quality. Most users start here — it's enough for a capable local chat assistant that runs entirely on your hardware.

What GPU should I get for 12.0 GB VRAM?

Popular GPUs with ~12.0 GB include Intel Arc B570, NVIDIA GeForce RTX 3080, NVIDIA GeForce RTX 2080 Ti. The NVIDIA GeForce RTX 3080 Ti leads in memory bandwidth at 912.4 GB/s, which translates directly to faster token generation. When choosing a GPU for AI, memory bandwidth matters as much as VRAM capacity — it determines how fast the model can generate text. A newer GPU with the same VRAM but higher bandwidth will produce tokens significantly faster.

Higher memory bandwidth = faster token generation. All these GPUs have approximately 12 GB VRAM, but speed varies significantly by bandwidth.

Memory bandwidth comparison

912.4 GB/s
760.3 GB/s
672 GB/s
616 GB/s
504 GB/s
How to choose the right model size for 12.0 GB?

The key rule: your model must fit in VRAM including KV cache overhead. With 12.0 GB, here's a practical guide: 7B models at Q4_K_M are your best bet — good quality and enough room for context. You can push to Q5_K_M for slightly better quality. 13B models barely fit at Q3, which works but quality suffers.

Should I get 12.0 GB or 16.0 GB for AI?

Upgrading from 12.0 GB to 16.0 GB gives you significantly more flexibility. At 12.0 GB you can run 1059 models; the jump to 16 GB is the biggest quality-of-life improvement — it opens up 14B models and lets you use higher quantizations on 7B models. If budget allows, the extra VRAM is always worth it for AI workloads — you can't add VRAM later.