GPU memory size · 13 cards·327 featured models fit

Best LLMs for 24 GB VRAM

Runs 27–31B and 35B MoE models well; 70B doesn't fit in 24 GB.

Speeds on

RTX 3090

RTX 4090103 tok/s RTX 3090 Ti103 tok/s RTX 309095.7 tok/s RX 7900 XTX90.6 tok/s

Top models by task

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 151 t/s	151 tok/s	Q4_K_M · 22.0 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.5 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 4090	Math Score	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s		Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s		Q4_K_M · 17.9 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 151 t/s	151 tok/s		Q4_K_M · 22.0 GB
4	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	overall	Q4_K_M · 21.7 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s		Q4_K_M · 19.3 GB

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
4	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 190 t/s	190 tok/s	Q4_K_M · 13.1 GB
5	LFM2.5 24B A2BLiquid AI Q4_K_M · 15.0 GB · 221 t/s	221 tok/s	Q4_K_M · 15.0 GB

#	Model	Speed on RTX 4090 ↓	Quant · VRAM
1	Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 201 t/s	201 tok/s	Q4_K_M · 3.4 GB
2	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 197 t/s	197 tok/s	Q4_K_M · 19.3 GB
3	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 190 t/s	190 tok/s	Q4_K_M · 13.1 GB
4	Qwen3 30B A3B Thinking 2507Alibaba Q4_K_M · 19.4 GB · 173 t/s	173 tok/s	Q4_K_M · 19.4 GB
5	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB

#	Model	Speed on RTX 4090	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 32.1 t/s	32.1 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 171 t/s	171 tok/s	Q4_K_M · 16.5 GB
5	JEV 27B VLautotrust Q4_K_M · 17.9 GB · 37.6 t/s	37.6 tok/s	Q4_K_M · 17.9 GB

See all 327 models that fit 24 GB →Best quant that fits · speeds on the RTX 4090

GPUs with 24 GB VRAM

Compare all GPUs →

Popular cards come first. Picks need a widely used card and a reasonable price.

Speed, price and value

NewUsed

Speed on Qwen3.5 9B (9B params) at Q4_K_M. New prices are compared with new, used with used; a card without a price for the view is shown at its MSRP. The last two columns are the speed you get per $100 of price and per 100 W of board power; higher is better.

Card	GB/s	New price	tok/s	tok/s per $100	tok/s per 100 W
RTX 4090NVIDIA · Ada · 2022 · 103 tok/sFastest	1,008	about $1,599	103	–	22.9
RTX 3090NVIDIA · Ampere · 2020 · 95.7 tok/s	936	about $1,499	95.7	–	27.3
RX 7900 XTXAMD · RDNA 3 · 2022 · 90.6 tok/sBest value newBest value used	960	about $1,130	90.6	8.0	25.5
RTX 3090 TiNVIDIA · Ampere · 2022 · 103 tok/s	1,008	about $1,999	103	–	22.9
9 more cards with 24 GB · less common or unpriced cards, not picked
RTX 5090 LaptopNVIDIA · Blackwell · 2025 · 91.6 tok/s	896	–	91.6	–	61.1
TITAN RTXNVIDIA · Turing · 2018 · 68.7 tok/s	672	about $2,499	68.7	–	24.5
Tesla P40NVIDIA · Pascal · 2016 · 35.4 tok/s	346	–	35.4	–	14.2
Pro B60Intel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $599	35.8	–	17.9
Tesla M40 24GBNVIDIA · Maxwell · 2015 · 29.4 tok/s	288	–	29.4	–	11.8
RTX PRO 4000 BlackwellNVIDIA · Blackwell · 2025 · 68.7 tok/s	672	about $1,500	68.7	–	47.4
RTX PRO 4000 Blackwell SFF EditionNVIDIA · Blackwell · 2025 · 44.2 tok/s	432	–	44.2	–	63.1
L4NVIDIA · Ada · 2023 · 30.7 tok/s	300	–	30.7	–	42.6
RTX A5000NVIDIA · Ampere · 2021 · 78.5 tok/s	768	about $2,250	78.5	–	34.1

Card	GB/s	Used price	tok/s	tok/s per $100	tok/s per 100 W
RTX 4090NVIDIA · Ada · 2022 · 103 tok/sFastest	1,008	about $3,000	103	3.4	22.9
RTX 3090NVIDIA · Ampere · 2020 · 95.7 tok/s	936	about $1,500	95.7	6.4	27.3
RX 7900 XTXAMD · RDNA 3 · 2022 · 90.6 tok/sBest value newBest value used	960	about $1,000	90.6	9.1	25.5
RTX 3090 TiNVIDIA · Ampere · 2022 · 103 tok/s	1,008	about $1,600	103	6.4	22.9
9 more cards with 24 GB · less common or unpriced cards, not picked
RTX 5090 LaptopNVIDIA · Blackwell · 2025 · 91.6 tok/s	896	–	91.6	–	61.1
TITAN RTXNVIDIA · Turing · 2018 · 68.7 tok/s	672	about $2,499	68.7	–	24.5
Tesla P40NVIDIA · Pascal · 2016 · 35.4 tok/s	346	–	35.4	–	14.2
Pro B60Intel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $599	35.8	–	17.9
Tesla M40 24GBNVIDIA · Maxwell · 2015 · 29.4 tok/s	288	–	29.4	–	11.8
RTX PRO 4000 BlackwellNVIDIA · Blackwell · 2025 · 68.7 tok/s	672	about $1,500	68.7	–	47.4
RTX PRO 4000 Blackwell SFF EditionNVIDIA · Blackwell · 2025 · 44.2 tok/s	432	–	44.2	–	63.1
L4NVIDIA · Ada · 2023 · 30.7 tok/s	300	–	30.7	–	42.6
RTX A5000NVIDIA · Ampere · 2021 · 78.5 tok/s	768	about $2,250	78.5	–	34.1

Nearby memory sizes

Next sizes down and up, plus why they're worth checking out. Speeds on Qwen3.5 9B.

8 GB less for the same money16 GB VRAMRuns up to 14B comfortably; 35B MoE only just fits

▼ 8 GB15 fewer modelssame speedRTX 5070 Ti$1,170rent $0.19/hr

Adds the 70B class; no single consumer card48 GB VRAMRuns up to 35B MoE; 70B only just fits

▲ 24 GB+5 models

Two 24 GB cards or a 64 GB Mac · up to 70B

Models that fit in 24 GB VRAM

Featured only

327 featured models · 242 run great. Best quant that fits with a 16K context, ordered by llmrun Score (100 = average model). Fit grades how well it runs, S best. How ranking works

Speeds on

LLM models compatible with RTX 3090 — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
Starcoder2 3B3.0BChatCode Q4_K_M·279.1 t/s	Q4_K_M	2.6 GB11%	279.1 t/s	16K	S100, Runs great
DeepSeek R1 Distill Llama 8B8.0BChatReasoningFunctions Q4_K_M·112.9 t/s	Q4_K_M	7.3 GB30%	112.9 t/s	131K	S94, Runs great
Phi 22.8BChatCode Q4_K_M·230.5 t/s	Q4_K_M	2.6 GB11%	230.5 t/s	2K	S100, Runs great
Aya Expanse 32B32.3BChat Q4_K_S·30.5 t/s	Q4_K_S	23.8 GB99%	30.5 t/s	—	C41, Usable
Gemma 3 12B Pt12.2BVision Q4_K_M·75.7 t/s	Q4_K_M	10.9 GB46%	75.7 t/s	—	S86, Runs great
Qwen3 Next 80B A3B Instruct81.3BChatFunctions IQ2_XXS·193.0 t/s	IQ2_XXS	22.9 GB95%	193.0 t/s	262K	B62, Decent
Qwen2.5 VL 32B Instruct33.5BVision Q4_K_S·31.0 t/s	Q4_K_S	23.4 GB98%	31.0 t/s	128K	C43, Usable
Phi 3.5 Vision Instruct4.1BVisionCode Q4_K_M·169.5 t/s	Q4_K_M	9.2 GB38%	169.5 t/s	131K	S98, Runs great
Mistral Nemo Instruct 240712.2BChatFunctions Q4_K_M·75.4 t/s	Q4_K_M	11.0 GB46%	75.4 t/s	131K	S86, Runs great
Internlm Chat 20B20BChat Q4_K_M·41.1 t/s	Q4_K_M	17.3 GB72%	41.1 t/s	4K	A75, Runs well
Qwen2.5 Coder 32B Instruct32.8BChatCodeFunctions Q4_K_S·31.6 t/s	Q4_K_S	23.0 GB96%	31.6 t/s	33K	B47, Decent
MiniCPM O 2 68.7BVisionFunctions Q4_K_M·108.3 t/s	Q4_K_M	6.4 GB27%	108.3 t/s	33K	S93, Runs great
Granite 3.1 2B Instruct2.5BChatFunctions Q4_K_M·305.8 t/s	Q4_K_M	3.2 GB13%	305.8 t/s	131K	S100, Runs great
Llama 2 13B Chat HF13.0BChat Q4_K_M·70.8 t/s	Q4_K_M	11.5 GB48%	70.8 t/s	—	S86, Runs great
Medgemma 27B IT27.4BVision Q4_K_M·33.6 t/s	Q4_K_M	21.7 GB91%	33.6 t/s	—	B56, Decent

What fits in 24 GB, and what doesn't

Tokens per second by model size. Grey rows need more than 24 GB.

9BQwen3.5 9B · Q4_K_M · 6.6 GB
103tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
69.0tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
37.6tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
151tok/s
70BLlama 3.3 70B Instruct · not at 4-bit
needs 51.3 GB
120B MoEGPT OSS 120B · not at 4-bit
needs 70.8 GB

Speed estimates for the RTX 4090: Q4_K_M, 4K context, single user. How we estimate

What the spare memory buys

Qwen3.8 27B at higher quants, and how much context still fits next to it.

Quant	Needs	Of 24 GB	Context
Q4_K_M	17.9 GB	74%	64K
Q5_K_M	21.0 GB	87%	64K

Bigger than 24 GB? Llama 3.3 70B Instruct needs 51.3 GB at Q4. Rent a bigger card by the hour, or step up a size.Rent H100 (80 GB), from $2.20/hr →48 GB cards →

24 GB on a Mac or mini PC

Macs keep 3.5 GB for the system, so a Mac needs 32 GB to match a 24 GB card (28.5 GB usable). They run the same models slower because memory bandwidth is lower.

Device	Usable	Bandwidth	Speed on Qwen3.8 27B
Mac Studio M1 Max (32 GB)M1 Max · 32 GB memory	28.5 GB	400 GB/s	16.1 tok/s
Mac Studio M2 Max (32 GB)M2 Max · 32 GB memory	28.5 GB	400 GB/s	16.1 tok/s
Mac Mini M2 Pro (32 GB)M2 Pro · 32 GB memory	28.5 GB	200 GB/s	8.0 tok/s
MacBook Pro 14" M1 Pro (32 GB)M1 Pro · 32 GB memory	28.5 GB	200 GB/s	8.0 tok/s

Best LLMs for a 32 GB Mac →27 more with this memory

24 GB VRAM FAQ

What models can I run with 24 GB of VRAM?

Models up to the 35B MoE class. 242 of 327 featured models run great. The top pick is Qwen3.8 27B at Q4_K_M: it needs 17.9 GB and runs at about 37.6 tok/s on the RTX 4090. Gemma 4 26B A4B IT and NVIDIA Nemotron 3.5 Lightning 30B A3B BF16 also run well.

Is 24 GB enough for 70B models?

No. Llama 3.3 70B Instruct needs 51.3 GB at Q4, more than 24 GB. Rent a bigger card by the hour, or look at 48 GB VRAM.

What is the best 24 GB GPU for local LLMs?

It depends on budget. Best value new and used: the RX 7900 XTX (90.6 tok/s); Fastest: the RTX 4090 (103 tok/s). Speeds are on Qwen3.5 9B.

How fast is a 24 GB card?

On the RTX 4090, Qwen3.5 9B runs at about 103 tok/s and Qwen3.8 27B at about 37.6 tok/s. Older and cheaper cards are slower: speed follows memory bandwidth, not memory size.

Is renting a 24 GB GPU cheaper than buying one?

An RTX 4090 rents from $0.34/hr. Renting costs as much as buying after ≈8,800 hours (≈6.0 years at 4 h a day).

Is a 24 GB Mac the same as a 24 GB GPU?

No. A Mac keeps about 3.5 GB for the system, so you need a 32 GB Mac to match. It is also slower: the Mac Studio M1 Max (32 GB) runs Qwen3.8 27B at about 16.1 tok/s.

Best coding LLMs Best GPUs for local LLMs by budget All GPUs ranked All hardware