llmrun

NVIDIA · Ada Lovelace

Best AI Models for NVIDIA RTX 4000 Ada Generation (20 GB)

Runs 14B models well; 27–31B fits but is slow (13 tok/s).

20 GB GDDR6360 GB/s107 TFLOPS FP16130 WAda Lovelace 2023

What LLMs can the RTX 4000 Ada Generation run?

#ModelSpeedllmrun ScoreQuant · VRAM
1
Gemma 4 26B A4B ITGoogleHow to run →
Q4_K_M · 16.5 GB · 74.7 t/s
74.7 tok/sQ4_K_M · 16.5 GB
2
Qwen3.5 9BAlibaba
Q4_K_M · 6.6 GB · 36.8 t/s
36.8 tok/sQ4_K_M · 6.6 GB
3
Qwen3.5 4BAlibaba
Q4_K_M · 3.4 GB · 71.8 t/s
71.8 tok/sQ4_K_M · 3.4 GB
4
GPT OSS 20BOpenAI
Q4_K_M · 13.1 GB · 80.7 t/s
80.7 tok/sQ4_K_M · 13.1 GB
5
LFM2.5 24B A2BLiquid AI
Q4_K_M · 15.0 GB · 115 t/s
115 tok/sQ4_K_M · 15.0 GB
See all 318 models on the RTX 4000 Ada Generation →Best quant that fits · speeds estimated on this card

About the RTX 4000 Ada Generation

The RTX 4000 Ada Generation runs 14B models well. Ada Lovelace generation, released 2023.

GenerationAda Lovelacereleased 2023

RTX 4000 Ada Generation strengths and weaknesses

Strengths

  • Runs 35B MoE models: Qwen3.6 35B A3B at 75.1 tok/s in 19.8 GB
  • Reads long prompts fast: A 4K-token prompt on Gemma 4 26B A4B IT starts answering in 749 ms
  • Low power draw: 130 W board power; a 300 W PSU is enough

Weaknesses

  • 70B models don't fit: Llama 3.3 70B Instruct needs 51.3 GB at Q4 vs 20 GB here
  • Slow on large dense models: Qwen3.8 27B at 13.4 tok/s

RTX 4000 Ada Generation tokens per second by model size

Output speed depends on memory bandwidth (360 GB/s). Grey rows need more than 20 GB.

  1. 9BQwen3.5 9B · Q4_K_M · 6.6 GB
    36.8tok/s
  2. 14BQwen3 14B · Q4_K_M · 11.9 GB
    24.6tok/s
  3. 27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
    13.4tok/s
  4. 35B MoEQwen3.6 35B A3B · IQ4_XS · 19.8 GB tight · IQ4_XS
    75.1tok/s
  5. 70BLlama 3.3 70B Instruct · not at 4-bit
    needs 51.3 GB
  6. 120B MoEGPT OSS 120B · not at 4-bit
    needs 70.8 GB

Estimated speeds: Q4_K_M, 4K context, one user. How we estimate

All 1117 models on the RTX 4000 Ada Generation

Featured only

Best quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works

LLM models compatible with NVIDIA RTX 4000 Ada Generation — ranked by Score, recency and Fit
ModelVRAMScoreFit
EXAONE 3.5 7.8B Instruct
Q4_K_M·45.3 t/s
7.7 GBA77, Runs well
Thinkless 1.5B RL DeepScaleR
Q4_K_M·164.8 t/s
1.8 GBS97, Runs great
Qwen3 8B 64k Context 2X Josiefied Uncensored
Q4_K_M·42.4 t/s
7.6 GBA76, Runs well
Qwen3.5 27B Claude 4.6 Opus Reasoning Distilled
Q4_K_M·13.4 t/s
17.9 GBC40, Usable
Qwen3 1.7B Abliterated
Q4_K_M·208.9 t/s
2.8 GBS100, Runs great
MediPhi Instruct
Q4_K_M·68.8 t/s
9.0 GBS85, Runs great
Qwen3.5 4B MiniFantasy MTP
FP16·23.9 t/s
10.0 GBB64, Decent
MobileLLM R1.5 950M
BF16·112.0 t/s
3.5 GBS94, Runs great
DeepSeek R1 Distill Qwen 32B
IQ3_M·15.0 t/s
19.3 GBD28, Barely runs
NVIDIA Nemotron 3 Nano 30B A3B BF16
Q4_K_M·81.8 t/s
19.3 GBB61, Decent
EXAONE 3.5 2.4B Instruct
Q4_K_M·147.2 t/s
3.4 GBS97, Runs great
Ornith 1.0 9B
Q4_K_M·37.7 t/s
6.5 GBA74, Runs well
Jan Code 4B
Q4_K_M·74.5 t/s
4.5 GBS86, Runs great
Qianfan OCR
Q4_K_M·70.3 t/s
4.7 GBS86, Runs great
Tiny GPT2LMHeadModel
Q4_K_M·23400.0 t/s
0.0 GBS100, Runs great

Rent NVIDIA RTX 4000 Ada Generation in the cloud

Hourly rate on demand for one RTX 4000 Ada Generation.

  • Vast.aiMarketplace

    from$0.19/hr

    Deploy

Buying one ($1,250) costs as much as ~6,600 hours of rental, about 28 months at 8 hours a day.

Cost per million tokens

Output-token cost on an RTX 4000 Ada Generation rented at $0.19/hr, one request at a time. Batch several users on the same card and the cost per token drops several-fold.

ModelSpeedPer 1M output
LFM2.5 24B A2B115 tok/s$0.45$0.0053 input
NVIDIA Nemotron 3.5 Lightning 30B A3B BF16111 tok/s$0.47$0.0051 input
GPT OSS 20B81 tok/s$0.65$0.0091 input
Gemma 4 26B A4B IT75 tok/s$0.70$0.0096 input
Qwen3.8 27B13 tok/s$3.90$0.060 input

RTX 4000 Ada Generation specs, power and PSU

Specs

Architecture
Ada Lovelace
VRAM
20 GB GDDR6
Memory bandwidth
360.0 GB/s
FP16 (dense)
106.90 TFLOPS
CUDA cores
6,144
Tensor cores
192
Compute capability
8.9
TDP
130 W
Released
2023-08-09
MSRP
$1,250

RTX 4000 Ada Generation FAQ

What is the largest model the RTX 4000 Ada Generation can run?

Qwen3.6 35B A3B (36.0B) is the largest at 4-bit or better: at IQ4_XS it uses 19.8 GB of 20 GB, and runs at about 75.1 tok/s.

Can the RTX 4000 Ada Generation run Gemma 4 26B A4B IT?

Yes. Gemma 4 26B A4B IT fits at Q4_K_M, using 16.5 GB of 20 GB, and it is the best model that runs well on this card.

Can the RTX 4000 Ada Generation run 70B models?

No. Llama 3.3 70B Instruct needs 51.3 GB at Q4, more than the 20 GB on this card.

How many tokens per second does the RTX 4000 Ada Generation give?

About 74.7 tok/s on Gemma 4 26B A4B IT at Q4_K_M, with the first token after 0.7 s on a 4K-token prompt.

What power supply does the RTX 4000 Ada Generation need?

Plan a 300 W PSU (2× the 130 W TDP).

Is 20 GB enough for coding and agents?

Yes for coding: Gemma 4 26B A4B IT, the best of the 3 coding models that fit, runs at about 75 tok/s. For agents, not yet: no agent-scored model fits in less than 96 GB.

Related

Best coding LLMs16 GB VRAM24 GB VRAMGPUs with 20 GB or moreRTX 5090RTX 3090 Ti