GPU memory size · 8 cards·368 featured models fit

Best LLMs for 48 GB VRAM

Runs 27–31B and 35B MoE models well; 70B only just fits in 48 GB.

Speeds onRTX 6000 Ada Generation

Top models by task

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 146 t/s	146 tok/s	Q4_K_M · 22.0 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.5 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Math Score	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s		Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s		Q4_K_M · 17.9 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 146 t/s	146 tok/s		Q4_K_M · 22.0 GB
4	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	overall	Q4_K_M · 21.7 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s		Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation ↓	Quant · VRAM
1	Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 191 t/s	191 tok/s	Q4_K_M · 3.4 GB
2	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB
3	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 183 t/s	183 tok/s	Q4_K_M · 13.1 GB
4	Qwen3 30B A3B Thinking 2507Alibaba Q4_K_M · 19.4 GB · 169 t/s	169 tok/s	Q4_K_M · 19.4 GB
5	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	MiniCPM V 4.7 35B A3BOpenBMB Q4_K_M · 22.1 GB · 163 t/s	163 tok/s	Q4_K_M · 22.1 GB

See all 368 models that fit 48 GB →Best quant that fits · speeds on the RTX 6000 Ada Generation

GPUs with 48 GB VRAM

Compare all GPUs →

Popular cards come first. Picks need a widely used card and a reasonable price.

Speed, price and value

NewUsed

Speed on Qwen3.5 9B (9B params) at Q4_K_M. New prices are compared with new, used with used; a card without a price for the view is shown at its MSRP. The last two columns are the speed you get per $100 of price and per 100 W of board power; higher is better.

Card	GB/s	New price	tok/s	tok/s per $100	tok/s per 100 W
8 more cards with 48 GB · less common or unpriced cards, not picked
Pro B60 Dual 48GBIntel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $1,200	35.8	–	8.9
L40SNVIDIA · Ada · 2023 · 88.3 tok/s	864	–	88.3	–	25.2
Quadro RTX 8000NVIDIA · Turing · 2018 · 68.7 tok/s	672	about $9,999	68.7	–	26.4
L40NVIDIA · Ada · 2022 · 88.3 tok/s	864	–	88.3	–	29.4
RTX 6000 Ada GenerationNVIDIA · Ada · 2022 · 98.1 tok/s	960	about $6,799	98.1	–	32.7
A40NVIDIA · Ampere · 2020 · 71.1 tok/s	696	–	71.1	–	23.7
PRO W7900AMD · RDNA 3 · 2023 · 81.5 tok/s	864	about $3,999	81.5	–	27.6
RTX A6000NVIDIA · Ampere · 2020 · 78.5 tok/s	768	about $4,649	78.5	–	26.2

Card	GB/s	Used price	tok/s	tok/s per $100	tok/s per 100 W
8 more cards with 48 GB · less common or unpriced cards, not picked
Pro B60 Dual 48GBIntel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $1,200	35.8	–	8.9
L40SNVIDIA · Ada · 2023 · 88.3 tok/s	864	–	88.3	–	25.2
Quadro RTX 8000NVIDIA · Turing · 2018 · 68.7 tok/s	672	about $9,999	68.7	–	26.4
L40NVIDIA · Ada · 2022 · 88.3 tok/s	864	–	88.3	–	29.4
RTX 6000 Ada GenerationNVIDIA · Ada · 2022 · 98.1 tok/s	960	about $6,799	98.1	–	32.7
A40NVIDIA · Ampere · 2020 · 71.1 tok/s	696	–	71.1	–	23.7
PRO W7900AMD · RDNA 3 · 2023 · 81.5 tok/s	864	about $3,999	81.5	–	27.6
RTX A6000NVIDIA · Ampere · 2020 · 78.5 tok/s	768	about $4,649	78.5	–	26.2

Nearby memory sizes

Next sizes down and up, plus why they're worth checking out. Speeds on Qwen3.5 9B.

24 GB less24 GB VRAMRuns up to 35B MoE comfortably

▼ 24 GB5 fewer modelsRX 7900 XTX$1,13090.6 tok/s

Models that fit in 48 GB VRAM

Featured only

1216 models. Best quant that fits with a 16K context, ordered by llmrun Score (100 = average model). Fit grades how well it runs, S best. How ranking works

LLM models compatible with RTX 6000 Ada Generation — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
OpenHermes 2.5 Mistral 7B7BChat Q4_K_M·130.8 t/s	Q4_K_M	6.7 GB14%	130.8 t/s	33K	S95, Runs great
GOT OCR2 0716MVision Q4_K_M·671.0 t/s	Q4_K_M	2.3 GB5%	671.0 t/s	33K	S100, Runs great
AliceAI Foundation 80B A3B Base81.3BChat Q4_K_S·131.1 t/s	Q4_K_S	46.8 GB98%	131.1 t/s	262K	C44, Usable
Dummy GPT2 Correct Vocab2MChat Q4_K_M·62400.0 t/s	Q4_K_M	0.0 GB0%	62400.0 t/s	1K	S100, Runs great
Openchat 3.5 12107.2BChat Q4_K_M·127.1 t/s	Q4_K_M	5.7 GB12%	127.1 t/s	8K	S95, Runs great
Qwen2.5 14B Instruct14.8BChatFunctions Q4_K_M·65.3 t/s	Q4_K_M	12.4 GB26%	65.3 t/s	33K	S85, Runs great
Pythia 1B1.1BChat Q4_K_M·878.9 t/s	Q4_K_M	0.7 GB1%	878.9 t/s	2K	S100, Runs great
T5 Paraphrase Paws223MChat Q4_K_M·4160.0 t/s	Q4_K_M	0.1 GB0%	4160.0 t/s	1K	S100, Runs great
NuExtract3.8BChat Q4_K_M·183.5 t/s	Q4_K_M	4.2 GB9%	183.5 t/s	4K	S99, Runs great
Openai GPT120MChat Q4_K_M·7800.0 t/s	Q4_K_M	0.1 GB0%	7800.0 t/s	1K	S100, Runs great
LFM2.5 1.2B Thinking1.2BChat Q4_K_M·583.2 t/s	Q4_K_M	1.2 GB3%	583.2 t/s	128K	S100, Runs great
Human Like LLama3 8B Instruct8.0BChat Q4_K_M·115.8 t/s	Q4_K_M	6.2 GB13%	115.8 t/s	8K	S94, Runs great
DialoGPT Small176MChat Q4_K_M·5200.0 t/s	Q4_K_M	0.1 GB0%	5200.0 t/s	1K	S100, Runs great
Dolphin 2.9 Llama3 8B8.0BChat Q4_K_M·115.8 t/s	Q4_K_M	6.2 GB13%	115.8 t/s	8K	S94, Runs great
Phi 1 51.4BChatCode Q4_K_M·663.8 t/s	Q4_K_M	0.9 GB2%	663.8 t/s	2K	S100, Runs great

What fits in 48 GB, and what doesn't

Tokens per second by model size. Grey rows need more than 48 GB.

9BQwen3.5 9B · Q4_K_M · 6.6 GB
98.1tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
65.7tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
35.8tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
146tok/s
70BLlama 3.3 70B Instruct · IQ4_XS · 46.4 GB tight · 97%
15.0tok/s
120B MoEGPT OSS 120B · not at 4-bit
needs 70.8 GB

Speed estimates for the RTX 6000 Ada Generation: Q4_K_M, 4K context, single user. How we estimate

What the spare memory buys

Qwen3.8 27B at higher quants, and how much context still fits next to it.

Quant	Needs	Of 48 GB	Context
Q4_K_M	17.9 GB	37%	256K
Q5_K_M	21.0 GB	44%	256K
Q6_K	24.1 GB	50%	256K
Q8_0	29.0 GB	60%	256K

Bigger than 48 GB? GPT OSS 120B needs 70.8 GB at Q4. Rent a bigger card by the hour, or look at a Mac with more memory.Rent RTX PRO 6000 Blackwell (96 GB), from $1.00/hr →

48 GB on a Mac or mini PC

Macs keep 3.5 GB for the system, so a Mac needs 64 GB to match a 48 GB card (60.5 GB usable). They run the same models slower because memory bandwidth is lower.

Device	Usable	Bandwidth	Speed on Qwen3.8 27B
Mac Pro M2 Ultra (64 GB)M2 Ultra · 64 GB memory	60.5 GB	800 GB/s	32.1 tok/s
Mac Studio M1 Ultra (64 GB)M1 Ultra · 64 GB memory	60.5 GB	800 GB/s	32.1 tok/s
Mac Studio M5 Max (64 GB)M5 Max · 64 GB memory	60.5 GB	614 GB/s	24.7 tok/s
Mac Studio M4 Max (64 GB)M4 Max · 64 GB memory	60.5 GB	546 GB/s	21.9 tok/s

Best LLMs for a 64 GB Mac →29 more with this memory

48 GB VRAM FAQ

What models can I run with 48 GB of VRAM?

Models up to the 70B class. 259 of 368 featured models run great. The top pick is Qwen3.8 27B at Q4_K_M: it needs 17.9 GB and runs at about 35.8 tok/s on the RTX 6000 Ada Generation. Qwen3.6 35B A3B and Gemma 4 26B A4B IT also run well.

Is 48 GB enough for 120B MoE models?

No. GPT OSS 120B needs 70.8 GB at Q4, more than 48 GB. Rent a bigger card by the hour, or look at a Mac with more memory.

How fast is a 48 GB card?

On the RTX 6000 Ada Generation, Qwen3.5 9B runs at about 98.1 tok/s and Qwen3.8 27B at about 35.8 tok/s. Older and cheaper cards are slower: speed follows memory bandwidth, not memory size.

Is a 48 GB Mac the same as a 48 GB GPU?

No. A Mac keeps about 3.5 GB for the system, so you need a 64 GB Mac to match. It is also slower: the Mac Pro M2 Ultra (64 GB) runs Qwen3.8 27B at about 32.1 tok/s.

Best coding LLMs Best GPUs for local LLMs by budget All GPUs ranked All hardware