GPU memory size · 8 cards·368 featured models fit

Best LLMs for 48 GB VRAM

Runs 27–31B and 35B MoE models well; 70B only just fits in 48 GB.

Speeds onRTX 6000 Ada Generation

Top models by task

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 146 t/s	146 tok/s	Q4_K_M · 22.0 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
3	Qwen3.5 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation	Math Score	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s		Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s		Q4_K_M · 17.9 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 146 t/s	146 tok/s		Q4_K_M · 22.0 GB
4	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	overall	Q4_K_M · 21.7 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s		Q4_K_M · 19.3 GB

#	Model	Speed on RTX 6000 Ada Generation ↓	Quant · VRAM
1	Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 191 t/s	191 tok/s	Q4_K_M · 3.4 GB
2	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 193 t/s	193 tok/s	Q4_K_M · 19.3 GB
3	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 183 t/s	183 tok/s	Q4_K_M · 13.1 GB
4	Qwen3 30B A3B Thinking 2507Alibaba Q4_K_M · 19.4 GB · 169 t/s	169 tok/s	Q4_K_M · 19.4 GB
5	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB

#	Model	Speed on RTX 6000 Ada Generation	Quant · VRAM
1	Qwen3.8 27BAlibabaHow to run → Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
2	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 35.8 t/s	35.8 tok/s	Q4_K_M · 17.9 GB
3	Gemma 4 31B ITGoogle Q4_K_M · 21.7 GB · 30.6 t/s	30.6 tok/s	Q4_K_M · 21.7 GB
4	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 165 t/s	165 tok/s	Q4_K_M · 16.5 GB
5	MiniCPM V 4.7 35B A3BOpenBMB Q4_K_M · 22.1 GB · 163 t/s	163 tok/s	Q4_K_M · 22.1 GB

See all 368 models that fit 48 GB →Best quant that fits · speeds on the RTX 6000 Ada Generation

GPUs with 48 GB VRAM

Compare all GPUs →

Popular cards come first. Picks need a widely used card and a reasonable price.

Speed, price and value

NewUsed

Speed on Qwen3.5 9B (9B params) at Q4_K_M. New prices are compared with new, used with used; a card without a price for the view is shown at its MSRP. The last two columns are the speed you get per $100 of price and per 100 W of board power; higher is better.

Card	GB/s	New price	tok/s	tok/s per $100	tok/s per 100 W
8 more cards with 48 GB · less common or unpriced cards, not picked
Pro B60 Dual 48GBIntel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $1,200	35.8	–	8.9
L40SNVIDIA · Ada · 2023 · 88.3 tok/s	864	–	88.3	–	25.2
Quadro RTX 8000NVIDIA · Turing · 2018 · 68.7 tok/s	672	about $9,999	68.7	–	26.4
L40NVIDIA · Ada · 2022 · 88.3 tok/s	864	–	88.3	–	29.4
RTX 6000 Ada GenerationNVIDIA · Ada · 2022 · 98.1 tok/s	960	about $6,799	98.1	–	32.7
A40NVIDIA · Ampere · 2020 · 71.1 tok/s	696	–	71.1	–	23.7
PRO W7900AMD · RDNA 3 · 2023 · 81.5 tok/s	864	about $3,999	81.5	–	27.6
RTX A6000NVIDIA · Ampere · 2020 · 78.5 tok/s	768	about $4,649	78.5	–	26.2

Card	GB/s	Used price	tok/s	tok/s per $100	tok/s per 100 W
8 more cards with 48 GB · less common or unpriced cards, not picked
Pro B60 Dual 48GBIntel · Xe2 (Battlemage) · 2025 · 35.8 tok/s	456	about $1,200	35.8	–	8.9
L40SNVIDIA · Ada · 2023 · 88.3 tok/s	864	–	88.3	–	25.2
Quadro RTX 8000NVIDIA · Turing · 2018 · 68.7 tok/s	672	about $9,999	68.7	–	26.4
L40NVIDIA · Ada · 2022 · 88.3 tok/s	864	–	88.3	–	29.4
RTX 6000 Ada GenerationNVIDIA · Ada · 2022 · 98.1 tok/s	960	about $6,799	98.1	–	32.7
A40NVIDIA · Ampere · 2020 · 71.1 tok/s	696	–	71.1	–	23.7
PRO W7900AMD · RDNA 3 · 2023 · 81.5 tok/s	864	about $3,999	81.5	–	27.6
RTX A6000NVIDIA · Ampere · 2020 · 78.5 tok/s	768	about $4,649	78.5	–	26.2

Nearby memory sizes

Next sizes down and up, plus why they're worth checking out. Speeds on Qwen3.5 9B.

24 GB less24 GB VRAMRuns up to 35B MoE comfortably

▼ 24 GB5 fewer modelsRX 7900 XTX$1,13090.6 tok/s

Models that fit in 48 GB VRAM

Featured only

1216 models. Best quant that fits with a 16K context, ordered by llmrun Score (100 = average model). Fit grades how well it runs, S best. How ranking works

LLM models compatible with RTX 6000 Ada Generation — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
PicoMistral 23M24MChat Q4_K_M·1950.0 t/s	Q4_K_M	0.3 GB1%	1950.0 t/s	1K	S100, Runs great
Falcon H1 7B Instruct7.6BChatFunctions Q4_K_M·123.8 t/s	Q4_K_M	6.3 GB13%	123.8 t/s	262K	S95, Runs great
Supra Mini V6 1M1MChat Q4_K_M·2080.0 t/s	Q4_K_M	0.3 GB1%	2080.0 t/s	1K	S100, Runs great
Qwen3 4B Hindi Instruct v24.0BChat Q4_K_M·215.2 t/s	Q4_K_M	4.2 GB9%	215.2 t/s	262K	S100, Runs great
GLM 4.7 Flash Heretic 1.2.029.9BChat Q4_K_M·167.2 t/s	Q4_K_M	19.1 GB40%	167.2 t/s	203K	S98, Runs great
Qwen3.5 9B Abliterated9.0BChatFunctions Q4_K_M·105.1 t/s	Q4_K_M	6.2 GB13%	105.1 t/s	262K	S92, Runs great
Sweep Next Edit v2 7B7.6BChatCode Q4_K_M·125.1 t/s	Q4_K_M	5.8 GB12%	125.1 t/s	33K	S95, Runs great
Kappa 20B 131k Mxfp420.9BChat Q4_K_M·182.8 t/s	Q4_K_M	13.1 GB27%	182.8 t/s	131K	S99, Runs great
GPT S 1.4M1MChat BF16·2012.9 t/s	BF16	0.3 GB1%	2012.9 t/s	0K	S100, Runs great
Hermes 3 Llama 3.1 8B8.0BChatRoleplayFunctions Q4_K_M·115.8 t/s	Q4_K_M	7.3 GB15%	115.8 t/s	131K	S94, Runs great
LFM2.5 1.2B Instruct Uncensored1.2BChat Q4_K_M·583.2 t/s	Q4_K_M	1.2 GB3%	583.2 t/s	128K	S100, Runs great
XCurOS0.1 8B Instruct7.6BChatFunctions BF16·39.9 t/s	BF16	16.5 GB34%	39.9 t/s	33K	A75, Runs well
OvisOCR2853MVisionFunctions Q4_K_M·725.6 t/s	Q4_K_M	0.9 GB2%	725.6 t/s	262K	S100, Runs great
Gemma 4 31B IT DFlash31BChat Q4_K_M·33.0 t/s	Q4_K_M	19.1 GB40%	33.0 t/s	262K	A72, Runs well
Sarvam 30B BF1632.2BChat Q4_K_M·182.9 t/s	Q4_K_M	19.9 GB41%	182.9 t/s	131K	S99, Runs great

What fits in 48 GB, and what doesn't

Tokens per second by model size. Grey rows need more than 48 GB.

9BQwen3.5 9B · Q4_K_M · 6.6 GB
98.1tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
65.7tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
35.8tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
146tok/s
70BLlama 3.3 70B Instruct · IQ4_XS · 46.4 GB tight · 97%
15.0tok/s
120B MoEGPT OSS 120B · not at 4-bit
needs 70.8 GB

Speed estimates for the RTX 6000 Ada Generation: Q4_K_M, 4K context, single user. How we estimate

What the spare memory buys

Qwen3.8 27B at higher quants, and how much context still fits next to it.

Quant	Needs	Of 48 GB	Context
Q4_K_M	17.9 GB	37%	256K
Q5_K_M	21.0 GB	44%	256K
Q6_K	24.1 GB	50%	256K
Q8_0	29.0 GB	60%	256K

Bigger than 48 GB? GPT OSS 120B needs 70.8 GB at Q4. Rent a bigger card by the hour, or look at a Mac with more memory.Rent RTX PRO 6000 Blackwell (96 GB), from $1.32/hr →

48 GB on a Mac or mini PC

Macs keep 3.5 GB for the system, so a Mac needs 64 GB to match a 48 GB card (60.5 GB usable). They run the same models slower because memory bandwidth is lower.

Device	Usable	Bandwidth	Speed on Qwen3.8 27B
Mac Pro M2 Ultra (64 GB)M2 Ultra · 64 GB memory	60.5 GB	800 GB/s	32.1 tok/s
Mac Studio M1 Ultra (64 GB)M1 Ultra · 64 GB memory	60.5 GB	800 GB/s	32.1 tok/s
Mac Studio M5 Max (64 GB)M5 Max · 64 GB memory	60.5 GB	614 GB/s	24.7 tok/s
Mac Studio M4 Max (64 GB)M4 Max · 64 GB memory	60.5 GB	546 GB/s	21.9 tok/s

Best LLMs for a 64 GB Mac →29 more with this memory

48 GB VRAM FAQ

What models can I run with 48 GB of VRAM?

Models up to the 70B class. 259 of 368 featured models run great. The top pick is Qwen3.8 27B at Q4_K_M: it needs 17.9 GB and runs at about 35.8 tok/s on the RTX 6000 Ada Generation. Qwen3.6 35B A3B and Gemma 4 26B A4B IT also run well.

Is 48 GB enough for 120B MoE models?

No. GPT OSS 120B needs 70.8 GB at Q4, more than 48 GB. Rent a bigger card by the hour, or look at a Mac with more memory.

How fast is a 48 GB card?

On the RTX 6000 Ada Generation, Qwen3.5 9B runs at about 98.1 tok/s and Qwen3.8 27B at about 35.8 tok/s. Older and cheaper cards are slower: speed follows memory bandwidth, not memory size.

Is a 48 GB Mac the same as a 48 GB GPU?

No. A Mac keeps about 3.5 GB for the system, so you need a 64 GB Mac to match. It is also slower: the Mac Pro M2 Ultra (64 GB) runs Qwen3.8 27B at about 32.1 tok/s.

Best coding LLMs Best GPUs for local LLMs by budget All GPUs ranked All hardware