Best local LLMs for vision

Run local LLMs that handle images, screenshots, and docs across your GPU or Mac. We rank them using the llmrun Composite Score and highlight our top choice for every VRAM tier.

Best for vision on

8 GB 12 GB 16 GB 24 GB 32 GB 48 GB+

Top pick

Gemma 4 26B A4B IT

Google · IQ3_S · 12 GB · up to 16K context · on RTX 5070

Score
Speed: 161.9 t/s
Grade: C

How to run →

All 96 vision models for 12 GB ↓

Also worth a look

Qwen3.5 9B

Q4_K_M · 6.6 GB · 68.7 t/s · 128K ctx

Gemma 3 12B IT

Q4_K_M · 10.9 GB · 54.3 t/s · 16K ctx

Also worth a look

Qwen3.5 9B

Q4_K_M · 6.6 GB · 68.7 t/s · 128K ctx

Best vision model by memory

Highest llmrun Composite Score that fits each size with at least 16K context and 10 t/s, at a Q3 or better quant. Speed is decode tokens per second on that size's reference GPU. The last row has no memory limit: the best model whatever it needs, on the GPU plan shown. Full ranking

8 GB · RTX 5060
Qwen3.5 9B45.8 t/s · 32K ctx
Q4_K_M · 6.6 GB
Runner-up: Gemma 3 4B IT
12 GB · RTX 5070
Gemma 4 26B A4B IT161.9 t/s · 16K ctx
IQ3_S · 12 GB
Runner-up: Qwen3.5 9B
16 GB · RTX 5080
Qwen3.8 27B41.6 t/s · 16K ctx
Q3_K_L · 15.4 GB
Runner-up: Gemma 4 26B A4B IT
24 GB · RTX 4090
Qwen3.8 27B37.6 t/s · 64K ctx
Q4_K_M · 17.9 GB
Runner-up: Gemma 4 31B IT
32 GB · RTX 5090
Qwen3.8 27B66.9 t/s · 256K ctx
Q4_K_M · 17.9 GB
Runner-up: Gemma 4 31B IT
48 GB+ · no limit · about 7× H100 SXM 80 GB
DeepSeek V4.1 Flash 10171M ctx
Q4_K_M · needs 458 GB
Runner-up: Kimi K3

Best vision model for each memory size
Memory	Best pick	Speed	Context	Runner-up
8 GBRTX 5060	Qwen3.5 9BQ4_K_M · 6.6 GB	45.8 t/s	32K	Gemma 3 4B IT64.0 · 102.5 t/s
12 GBRTX 5070	Gemma 4 26B A4B ITIQ3_S · 12 GB	161.9 t/s	16K	Qwen3.5 9B99.4 · 68.7 t/s
16 GBRTX 5080	Qwen3.8 27BQ3_K_L · 15.4 GB	41.6 t/s	16K	Gemma 4 26B A4B IT105.8 · 172.8 t/s
24 GBRTX 4090	Qwen3.8 27BQ4_K_M · 17.9 GB	37.6 t/s	64K	Gemma 4 31B IT106.6 · 32.1 t/s
32 GBRTX 5090	Qwen3.8 27BQ4_K_M · 17.9 GB	66.9 t/s	256K	Gemma 4 31B IT106.6 · 57.1 t/s
48 GB+ · no limitabout 7× H100 SXM 80 GB	DeepSeek V4.1 Flash 1017Q4_K_M · needs 458 GB	n/a	1M	Kimi K3132.8

Vision models that run on 12 GB96

Ranked by Composite Score, models without one after. Best quant that fits with a 16K context; speed is decode tokens per second on the reference GPU.

96 models · 89 run great · 3 run well

Featured only

Score: llmrun Score, 100 = average model. Fit: how well it runs on this hardware, S best to F won't fit. Ordered by Score, recency and Fit. How ranking works

Showing compatibility for NVIDIA GeForce RTX 5070

LLM models compatible with NVIDIA GeForce RTX 5070 — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
Jina Ocr V13.4BVision Q4_K_M·382.0 t/s	Q4_K_M	3.3 GB28%	382.0 t/s	33K	S100, Runs great
LightOnOCR 3 1B1.0BVision Q4_K_M·428.2 t/s	Q4_K_M	1.8 GB15%	428.2 t/s	16K	S100, Runs great
Qwen3 VL 2B Instruct2.1BVisionFunctions Q4_K_M·241.3 t/s	Q4_K_M	3.5 GB29%	241.3 t/s	262K	S100, Runs great
HunyuanOCR1.1BVision Q4_K_M·408.2 t/s	Q4_K_M	1.8 GB15%	408.2 t/s	131K	S100, Runs great
Qianfan OCR4.7BVision Q4_K_M·131.2 t/s	Q4_K_M	4.7 GB39%	131.2 t/s	33K	S95, Runs great
Medgemma 1.5 4B IT4.3BVision Q4_K_M·153.8 t/s	Q4_K_M	5 GB42%	153.8 t/s	—	S97, Runs great
Idefics3 8B Llama38.5BVision Q4_K_M·77.3 t/s	Q4_K_M	7.5 GB63%	77.3 t/s	131K	S87, Runs great
Llama 3.2 11B Vision Instruct10.7BVisionFunctions Q4_K_M·62.0 t/s	Q4_K_M	9.8 GB82%	62.0 t/s	—	A82, Runs well
H2ovl Mississippi 2B2.2BChatVision BF16·92.2 t/s	BF16	6.5 GB54%	92.2 t/s	—	S90, Runs great
LFM2 VL 1.6B1.6BVision Q4_K_M·330.9 t/s	Q4_K_M	1.5 GB12%	330.9 t/s	128K	S100, Runs great
Dots.ocr3.0BVision Q4_K_M·200.4 t/s	Q4_K_M	2.6 GB22%	200.4 t/s	131K	S100, Runs great
Qwen2.5 VL 7B Instruct8.3BVision Q4_K_M·81.0 t/s	Q4_K_M	6.2 GB52%	81.0 t/s	128K	S88, Runs great
Translategemma 4B IT4.3BVision Q4_K_M·153.8 t/s	Q4_K_M	5 GB42%	153.8 t/s	—	S97, Runs great
Gemma 3n E4B IT7.8BVision Q4_K_M·84.3 t/s	Q4_K_M	7.7 GB64%	84.3 t/s	—	S88, Runs great
Qwen2.5 VL 3B Instruct3.8BVision Q4_K_M·166.1 t/s	Q4_K_M	3.2 GB26%	166.1 t/s	128K	S98, Runs great

How much memory do you need for a vision model?

On 8 GB the best vision model is Qwen3.5 9B (score 99). At 12 GB it is Gemma 4 26B A4B IT (score 106). From 16 to 32 GB it is Qwen3.8 27B (score 122). With no memory limit, the best is DeepSeek V4.1 Flash 1017 (score 133). Add a bit more breathing room to your setup when using a vision model with an image encoder alongside the language model; each image loads as extra tokens in the context. Allocate slightly more space than you would for a text-only model of equal size.

Frequently Asked Questions

What is the best local LLM for vision?

On 24 GB, Qwen3.8 27B is the top pick with a score of 122, running at about 38 t/s on RTX 4090. It fits up to 64K tokens of context there. On 8 GB, the best fit is Qwen3.5 9B. With no memory limit, the best is DeepSeek V4.1 Flash 1017. Open a model to see which GPUs and Macs run it.

Can I run a vision model on a laptop or an 8 GB GPU?

Yes. On 8 GB the best pick is Qwen3.5 9B (Q4_K_M, 6.6 GB), which runs at about 46 t/s on RTX 5060. Laptops with 8 to 16 GB of memory can run smaller models in the same way.

Can local models read screenshots and documents?

Vision-language models take an image along with your prompt to answer questions, pull out text, or describe charts. Whether support works depends on the runtime; however, you can run them in Ollama or LM Studio.

Do images need extra memory?

The image encoder bumps up the model size. Each picture turns into hundreds of tokens within the context window, swelling the cache. Bigger images demand even more.