GPU memory size · 8 cards·368 featured models fit

Best LLMs for 48 GB VRAM

Runs 27–31B and 35B MoE models well; 70B only just fits in 48 GB.

Speeds onRTX 6000 Ada Generation

Top models by task

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 146 t/s	146 tok/s	Q4_K_M · 22.0 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.5 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Math Score	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s		Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s		Q4_K_M · 17.9 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 146 t/s	146 tok/s		Q4_K_M · 22.0 GB
4	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	overall	Q4_K_M · 21.7 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s		Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation ↓	Quant · VRAM
1	Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 191 t/s	191 tok/s	Q4_K_M · 3.4 GB
2	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB
3	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 183 t/s	183 tok/s	Q4_K_M · 13.1 GB
4	Qwen3 30B A3B Thinking 2507Alibaba Q4_K_M · 19.4 GB · 169 t/s	169 tok/s	Q4_K_M · 19.4 GB
5	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	MiniCPM V 4.7 35B A3BOpenBMB Q4_K_M · 22.1 GB · 163 t/s	163 tok/s	Q4_K_M · 22.1 GB

See all 368 models that fit 48 GB →Best quant that fits · speeds on the RTX 6000 Ada Generation

GPUs with 48 GB VRAM

Compare all GPUs →

Popular cards come first. Picks need a widely used card and a reasonable price.

Speed, price and value

NewUsed

Speed on Qwen3.5 9B (9B params) at Q4_K_M. New prices are compared with new, used with used; a card without a price for the view is shown at its MSRP. The last two columns are the speed you get per $100 of price and per 100 W of board power; higher is better.

Card	GB/s	New price	tok/s	tok/s per $100	tok/s per 100 W
8 more cards with 48 GB · less common or unpriced cards, not picked
Pro B60 Dual 48GBIntel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $1,200	35.8	–	8.9
L40SNVIDIA · Ada · 2023 · 88.3 tok/s	864	–	88.3	–	25.2
Quadro RTX 8000NVIDIA · Turing · 2018 · 68.7 tok/s	672	about $9,999	68.7	–	26.4
L40NVIDIA · Ada · 2022 · 88.3 tok/s	864	–	88.3	–	29.4
RTX 6000 Ada GenerationNVIDIA · Ada · 2022 · 98.1 tok/s	960	about $6,799	98.1	–	32.7
A40NVIDIA · Ampere · 2020 · 71.1 tok/s	696	–	71.1	–	23.7
PRO W7900AMD · RDNA 3 · 2023 · 81.5 tok/s	864	about $3,999	81.5	–	27.6
RTX A6000NVIDIA · Ampere · 2020 · 78.5 tok/s	768	about $4,649	78.5	–	26.2

Card	GB/s	Used price	tok/s	tok/s per $100	tok/s per 100 W
8 more cards with 48 GB · less common or unpriced cards, not picked
Pro B60 Dual 48GBIntel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $1,200	35.8	–	8.9
L40SNVIDIA · Ada · 2023 · 88.3 tok/s	864	–	88.3	–	25.2
Quadro RTX 8000NVIDIA · Turing · 2018 · 68.7 tok/s	672	about $9,999	68.7	–	26.4
L40NVIDIA · Ada · 2022 · 88.3 tok/s	864	–	88.3	–	29.4
RTX 6000 Ada GenerationNVIDIA · Ada · 2022 · 98.1 tok/s	960	about $6,799	98.1	–	32.7
A40NVIDIA · Ampere · 2020 · 71.1 tok/s	696	–	71.1	–	23.7
PRO W7900AMD · RDNA 3 · 2023 · 81.5 tok/s	864	about $3,999	81.5	–	27.6
RTX A6000NVIDIA · Ampere · 2020 · 78.5 tok/s	768	about $4,649	78.5	–	26.2

Nearby memory sizes

Next sizes down and up, plus why they're worth checking out. Speeds on Qwen3.5 9B.

24 GB less24 GB VRAMRuns up to 35B MoE comfortably

▼ 24 GB5 fewer modelsRX 7900 XTX$1,13090.6 tok/s

Models that fit in 48 GB VRAM

Featured only

1216 models. Best quant that fits with a 16K context, ordered by llmrun Score (100 = average model). Fit grades how well it runs, S best. How ranking works

LLM models compatible with RTX 6000 Ada Generation — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
Falcon Mamba 7B7.3BChat Q4_K_M·130.0 t/s	Q4_K_M	7.3 GB15%	130.0 t/s	—	S95, Runs great
GLM 4.5V107.7BVisionFunctions IQ3_S·81.0 t/s	IQ3_S	47.1 GB98%	81.0 t/s	66K	C44, Usable
Apertus 70B 250970.6BChat Q3_K_M·17.6 t/s	Q3_K_M	40.1 GB84%	17.6 t/s	66K	B52, Decent
Mistral Small 4 119B 2603119.4BChatFunctions IQ2_S·176.7 t/s	IQ2_S	47.3 GB99%	176.7 t/s	1049K	C44, Usable
Mellum 4B Base4.0BChatCode Q4_K_M·179.8 t/s	Q4_K_M	5.7 GB12%	179.8 t/s	8K	S99, Runs great
Phi 3 Medium 128k Instruct14.0BChatCode Q4_K_M·68.6 t/s	Q4_K_M	12.0 GB25%	68.6 t/s	131K	S85, Runs great
Meta Llama 3 8B8.0BChat Q4_K_M·115.8 t/s	Q4_K_M	6.2 GB13%	115.8 t/s	8K	S94, Runs great
Phi 3 Mini 128k Instruct3.8BChatCode Q4_K_M·183.5 t/s	Q4_K_M	9.0 GB19%	183.5 t/s	131K	S99, Runs great
Qwen2.5 72B Instruct Abliterated72.7BChat Q4_K_S·14.9 t/s	Q4_K_S	46.6 GB97%	14.9 t/s	33K	D28, Barely runs
StableBeluga270BChat Q4_K_M·14.5 t/s	Q4_K_M	43.6 GB91%	14.5 t/s	4K	C38, Usable
Mistral 7B Instruct v0.27.2BChat Q4_K_M·127.1 t/s	Q4_K_M	6.8 GB14%	127.1 t/s	33K	S95, Runs great
Macbert4csc Base Chinese102MChat BF16·2713.0 t/s	BF16	0.2 GB0%	2713.0 t/s	1K	S100, Runs great
SmolLM 1.7B1.7BChat Q4_K_M·360.7 t/s	Q4_K_M	1.7 GB4%	360.7 t/s	2K	S100, Runs great
OpenReasoning Nemotron 32B32.8BChatCodeReasoning Q4_K_M·30.4 t/s	Q4_K_M	24.3 GB51%	30.4 t/s	131K	A71, Runs well
H2ogpt 4096 Llama2 70B Chat69.0BChat Q4_K_M·14.7 t/s	Q4_K_M	43.0 GB90%	14.7 t/s	4K	C40, Usable

What fits in 48 GB, and what doesn't

Tokens per second by model size. Grey rows need more than 48 GB.

9BQwen3.5 9B · Q4_K_M · 6.6 GB
98.1tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
65.7tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
35.8tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
146tok/s
70BLlama 3.3 70B Instruct · IQ4_XS · 46.4 GB tight · 97%
15.0tok/s
120B MoEGPT OSS 120B · not at 4-bit
needs 70.8 GB

Speed estimates for the RTX 6000 Ada Generation: Q4_K_M, 4K context, single user. How we estimate

What the spare memory buys

Qwen3.8 27B at higher quants, and how much context still fits next to it.

Quant	Needs	Of 48 GB	Context
Q4_K_M	17.9 GB	37%	256K
Q5_K_M	21.0 GB	44%	256K
Q6_K	24.1 GB	50%	256K
Q8_0	29.0 GB	60%	256K

Bigger than 48 GB? GPT OSS 120B needs 70.8 GB at Q4. Rent a bigger card by the hour, or look at a Mac with more memory.Rent RTX PRO 6000 Blackwell (96 GB), from $1.52/hr →

48 GB on a Mac or mini PC

Macs keep 3.5 GB for the system, so a Mac needs 64 GB to match a 48 GB card (60.5 GB usable). They run the same models slower because memory bandwidth is lower.

Device	Usable	Bandwidth	Speed on Qwen3.8 27B
Mac Pro M2 Ultra (64 GB)M2 Ultra · 64 GB memory	60.5 GB	800 GB/s	32.1 tok/s
Mac Studio M1 Ultra (64 GB)M1 Ultra · 64 GB memory	60.5 GB	800 GB/s	32.1 tok/s
Mac Studio M5 Max (64 GB)M5 Max · 64 GB memory	60.5 GB	614 GB/s	24.7 tok/s
Mac Studio M4 Max (64 GB)M4 Max · 64 GB memory	60.5 GB	546 GB/s	21.9 tok/s

Best LLMs for a 64 GB Mac →29 more with this memory

48 GB VRAM FAQ

What models can I run with 48 GB of VRAM?

Models up to the 70B class. 259 of 368 featured models run great. The top pick is Qwen3.8 27B at Q4_K_M: it needs 17.9 GB and runs at about 35.8 tok/s on the RTX 6000 Ada Generation. Qwen3.6 35B A3B and Gemma 4 26B A4B IT also run well.

Is 48 GB enough for 120B MoE models?

No. GPT OSS 120B needs 70.8 GB at Q4, more than 48 GB. Rent a bigger card by the hour, or look at a Mac with more memory.

How fast is a 48 GB card?

On the RTX 6000 Ada Generation, Qwen3.5 9B runs at about 98.1 tok/s and Qwen3.8 27B at about 35.8 tok/s. Older and cheaper cards are slower: speed follows memory bandwidth, not memory size.

Is a 48 GB Mac the same as a 48 GB GPU?

No. A Mac keeps about 3.5 GB for the system, so you need a 64 GB Mac to match. It is also slower: the Mac Pro M2 Ultra (64 GB) runs Qwen3.8 27B at about 32.1 tok/s.

Best coding LLMs Best GPUs for local LLMs by budget All GPUs ranked All hardware