GPU memory size · 13 cards·327 featured models fit

Best LLMs for 24 GB VRAM

Runs 27–31B and 35B MoE models well; 70B doesn't fit in 24 GB.

Speeds on

RTX 3090

RTX 4090103 tok/s RTX 3090 Ti103 tok/s RTX 309095.7 tok/s RX 7900 XTX90.6 tok/s

Top models by task

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 151 t/s	151 tok/s	Q4_K_M · 22.0 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.5 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 4090	Math Score	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s		Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s		Q4_K_M · 17.9 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 151 t/s	151 tok/s		Q4_K_M · 22.0 GB
4	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	overall	Q4_K_M · 21.7 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s		Q4_K_M · 19.3 GB

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
4	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 190 t/s	190 tok/s	Q4_K_M · 13.1 GB
5	LFM2.5 24B A2BLiquid AI Q4_K_M · 15.0 GB · 221 t/s	221 tok/s	Q4_K_M · 15.0 GB

#	Model	Speed on RTX 4090 ↓	Quant · VRAM
1	Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 201 t/s	201 tok/s	Q4_K_M · 3.4 GB
2	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s	Q4_K_M · 19.3 GB
3	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 190 t/s	190 tok/s	Q4_K_M · 13.1 GB
4	Qwen3 30B A3B Thinking 2507Alibaba Q4_K_M · 19.4 GB · 173 t/s	173 tok/s	Q4_K_M · 19.4 GB
5	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
5	JEV 27B VLautotrust Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB

See all 327 models that fit 24 GB →Best quant that fits · speeds on the RTX 4090

GPUs with 24 GB VRAM

Compare all GPUs →

Popular cards come first. Picks need a widely used card and a reasonable price.

Speed, price and value

NewUsed

Speed on Qwen3.5 9B (9B params) at Q4_K_M. New prices are compared with new, used with used; a card without a price for the view is shown at its MSRP. The last two columns are the speed you get per $100 of price and per 100 W of board power; higher is better.

Card	GB/s	New price	tok/s	tok/s per $100	tok/s per 100 W
RTX 4090NVIDIA · Ada · 2022 · 103 tok/sFastest	1,008	about $1,599	103	–	22.9
RTX 3090NVIDIA · Ampere · 2020 · 95.7 tok/s	936	about $1,499	95.7	–	27.3
RX 7900 XTXAMD · RDNA 3 · 2022 · 90.6 tok/sBest value newBest value used	960	about $1,130	90.6	8.0	25.5
RTX 3090 TiNVIDIA · Ampere · 2022 · 103 tok/s	1,008	about $1,999	103	–	22.9
9 more cards with 24 GB · less common or unpriced cards, not picked
RTX 5090 LaptopNVIDIA · Blackwell · 2025 · 91.6 tok/s	896	–	91.6	–	61.1
TITAN RTXNVIDIA · Turing · 2018 · 68.7 tok/s	672	about $2,499	68.7	–	24.5
Tesla P40NVIDIA · Pascal · 2016 · 35.4 tok/s	346	–	35.4	–	14.2
Pro B60Intel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $599	35.8	–	17.9
Tesla M40 24GBNVIDIA · Maxwell · 2015 · 29.4 tok/s	288	–	29.4	–	11.8
RTX PRO 4000 BlackwellNVIDIA · Blackwell · 2025 · 68.7 tok/s	672	about $1,500	68.7	–	47.4
RTX PRO 4000 Blackwell SFF EditionNVIDIA · Blackwell · 2025 · 44.2 tok/s	432	–	44.2	–	63.1
L4NVIDIA · Ada · 2023 · 30.7 tok/s	300	–	30.7	–	42.6
RTX A5000NVIDIA · Ampere · 2021 · 78.5 tok/s	768	about $2,250	78.5	–	34.1

Card	GB/s	Used price	tok/s	tok/s per $100	tok/s per 100 W
RTX 4090NVIDIA · Ada · 2022 · 103 tok/sFastest	1,008	about $3,000	103	3.4	22.9
RTX 3090NVIDIA · Ampere · 2020 · 95.7 tok/s	936	about $1,500	95.7	6.4	27.3
RX 7900 XTXAMD · RDNA 3 · 2022 · 90.6 tok/sBest value newBest value used	960	about $1,000	90.6	9.1	25.5
RTX 3090 TiNVIDIA · Ampere · 2022 · 103 tok/s	1,008	about $1,600	103	6.4	22.9
9 more cards with 24 GB · less common or unpriced cards, not picked
RTX 5090 LaptopNVIDIA · Blackwell · 2025 · 91.6 tok/s	896	–	91.6	–	61.1
TITAN RTXNVIDIA · Turing · 2018 · 68.7 tok/s	672	about $2,499	68.7	–	24.5
Tesla P40NVIDIA · Pascal · 2016 · 35.4 tok/s	346	–	35.4	–	14.2
Pro B60Intel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $599	35.8	–	17.9
Tesla M40 24GBNVIDIA · Maxwell · 2015 · 29.4 tok/s	288	–	29.4	–	11.8
RTX PRO 4000 BlackwellNVIDIA · Blackwell · 2025 · 68.7 tok/s	672	about $1,500	68.7	–	47.4
RTX PRO 4000 Blackwell SFF EditionNVIDIA · Blackwell · 2025 · 44.2 tok/s	432	–	44.2	–	63.1
L4NVIDIA · Ada · 2023 · 30.7 tok/s	300	–	30.7	–	42.6
RTX A5000NVIDIA · Ampere · 2021 · 78.5 tok/s	768	about $2,250	78.5	–	34.1

Nearby memory sizes

Next sizes down and up, plus why they're worth checking out. Speeds on Qwen3.5 9B.

8 GB less for the same money16 GB VRAMRuns up to 14B comfortably; 35B MoE only just fits

▼ 8 GB15 fewer modelssame speedRTX 5070 Ti$1,170rent $0.18/hr

Adds the 70B class; no single consumer card48 GB VRAMRuns up to 35B MoE; 70B only just fits

▲ 24 GB+5 models

Two 24 GB cards or a 64 GB Mac · up to 70B

Models that fit in 24 GB VRAM

Featured only

327 featured models · 242 run great. Best quant that fits with a 16K context, ordered by llmrun Score (100 = average model). Fit grades how well it runs, S best. How ranking works

Speeds on

LLM models compatible with RTX 3090 — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
Idefics3 8B Llama38.5BVision Q4_K_M·107.7 t/s	Q4_K_M	7.5 GB31%	107.7 t/s	131K	S93, Runs great
Eurus 2 7B PRIME7.6BChat Q4_K_M·121.9 t/s	Q4_K_M	5.1 GB21%	121.9 t/s	4K	S95, Runs great
Mathstral 7B v0.17.2BMath Q4_K_M·123.7 t/s	Q4_K_M	6.8 GB28%	123.7 t/s	33K	S95, Runs great
INTELLECT 1 Instruct10.2BChat Q4_K_M·89.8 t/s	Q4_K_M	7.8 GB33%	89.8 t/s	8K	S89, Runs great
GLM Z1 32B 041432.6BChatFunctions Q4_K_M·30.5 t/s	Q4_K_M	20.9 GB87%	30.5 t/s	33K	B62, Decent
Magistral Small 250623.6BChat Q4_K_M·41.0 t/s	Q4_K_M	17.8 GB74%	41.0 t/s	41K	A75, Runs well
EXAONE 4.5 33B34.4BVisionFunctions Q4_K_S·30.2 t/s	Q4_K_S	23.9 GB100%	30.2 t/s	262K	C39, Usable
Qwen2.5 Omni 3B5.5BChat Q4_K_M·166.7 t/s	Q4_K_M	6.0 GB25%	166.7 t/s	—	S98, Runs great
Qwen3 VL 2B Instruct2.1BVisionFunctions Q4_K_M·336.2 t/s	Q4_K_M	3.5 GB14%	336.2 t/s	262K	S100, Runs great
TinyLlama 1.1B Chat v1.01.1BChat Q4_K_M·602.5 t/s	Q4_K_M	1.0 GB4%	602.5 t/s	2K	S100, Runs great
Qwen3 VL 32B Instruct33.4BVisionFunctions Q4_K_M·29.5 t/s	Q4_K_M	23 GB96%	29.5 t/s	262K	B46, Decent
Qwen3 32B32.8BChatFunctions Q4_K_M·30.0 t/s	Q4_K_M	22.6 GB94%	30.0 t/s	41K	B50, Decent
Dots.ocr3.0BVision Q4_K_M·279.1 t/s	Q4_K_M	2.6 GB11%	279.1 t/s	131K	S100, Runs great
Moondream3 Preview9.3BVision BF16·29.8 t/s	BF16	23.1 GB96%	29.8 t/s	—	B46, Decent
Translategemma 4B IT4.3BVision Q4_K_M·214.3 t/s	Q4_K_M	5 GB21%	214.3 t/s	—	S100, Runs great

What fits in 24 GB, and what doesn't

Tokens per second by model size. Grey rows need more than 24 GB.

9BQwen3.5 9B · Q4_K_M · 6.6 GB
103tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
69.0tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
37.6tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
151tok/s
70BLlama 3.3 70B Instruct · not at 4-bit
needs 51.3 GB
120B MoEGPT OSS 120B · not at 4-bit
needs 70.8 GB

Speed estimates for the RTX 4090: Q4_K_M, 4K context, single user. How we estimate

What the spare memory buys

Qwen3.8 27B at higher quants, and how much context still fits next to it.

Quant	Needs	Of 24 GB	Context
Q4_K_M	17.9 GB	74%	64K
Q5_K_M	21.0 GB	87%	64K

Bigger than 24 GB? Llama 3.3 70B Instruct needs 51.3 GB at Q4. Rent a bigger card by the hour, or step up a size.Rent A100 (80 GB), from $1.18/hr →48 GB cards →

24 GB on a Mac or mini PC

Macs keep 3.5 GB for the system, so a Mac needs 32 GB to match a 24 GB card (28.5 GB usable). They run the same models slower because memory bandwidth is lower.

Device	Usable	Bandwidth	Speed on Qwen3.8 27B
Mac Studio M1 Max (32 GB)M1 Max · 32 GB memory	28.5 GB	400 GB/s	16.1 tok/s
Mac Studio M2 Max (32 GB)M2 Max · 32 GB memory	28.5 GB	400 GB/s	16.1 tok/s
Mac Mini M2 Pro (32 GB)M2 Pro · 32 GB memory	28.5 GB	200 GB/s	8.0 tok/s
MacBook Pro 14" M1 Pro (32 GB)M1 Pro · 32 GB memory	28.5 GB	200 GB/s	8.0 tok/s

Best LLMs for a 32 GB Mac →27 more with this memory

24 GB VRAM FAQ

What models can I run with 24 GB of VRAM?

Models up to the 35B MoE class. 242 of 327 featured models run great. The top pick is Qwen3.8 27B at Q4_K_M: it needs 17.9 GB and runs at about 37.6 tok/s on the RTX 4090. Gemma 4 26B A4B IT and NVIDIA Nemotron 3.5 Lightning 30B A3B BF16 also run well.

Is 24 GB enough for 70B models?

No. Llama 3.3 70B Instruct needs 51.3 GB at Q4, more than 24 GB. Rent a bigger card by the hour, or look at 48 GB VRAM.

What is the best 24 GB GPU for local LLMs?

It depends on budget. Best value new and used: the RX 7900 XTX (90.6 tok/s); Fastest: the RTX 4090 (103 tok/s). Speeds are on Qwen3.5 9B.

How fast is a 24 GB card?

On the RTX 4090, Qwen3.5 9B runs at about 103 tok/s and Qwen3.8 27B at about 37.6 tok/s. Older and cheaper cards are slower: speed follows memory bandwidth, not memory size.

Is renting a 24 GB GPU cheaper than buying one?

An RTX 4090 rents from $0.34/hr. Renting costs as much as buying after ≈8,800 hours (≈6.0 years at 4 h a day).

Is a 24 GB Mac the same as a 24 GB GPU?

No. A Mac keeps about 3.5 GB for the system, so you need a 32 GB Mac to match. It is also slower: the Mac Studio M1 Max (32 GB) runs Qwen3.8 27B at about 16.1 tok/s.

Best coding LLMs Best GPUs for local LLMs by budget All GPUs ranked All hardware