NVIDIA · Ada Lovelace
Best AI Models for NVIDIA RTX 4000 Ada Generation (20 GB)
Runs 14B models well; 27–31B fits but is slow (13 tok/s).
| # | Model | Speed | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 16.5 GB · 74.7 t/s | 74.7 tok/s | Q4_K_M · 16.5 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 36.8 t/s | 36.8 tok/s | Q4_K_M · 6.6 GB | |
| 3 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 71.8 t/s | 71.8 tok/s | Q4_K_M · 3.4 GB | |
| 4 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 80.7 t/s | 80.7 tok/s | Q4_K_M · 13.1 GB | |
| 5 | LFM2.5 24B A2BLiquid AI Q4_K_M · 15.0 GB · 115 t/s | 115 tok/s | Q4_K_M · 15.0 GB |
| # | Model | Speed | Coding Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 16.5 GB · 74.7 t/s | 74.7 tok/s | Q4_K_M · 16.5 GB | |
| 2 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 80.7 t/s | 80.7 tok/s | Q4_K_M · 13.1 GB | |
| 3 | Q4_K_M · 7.9 GB · 44.2 t/s | 44.2 tok/s | Q4_K_M · 7.9 GB | |
| 4 | Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 13.4 t/s | 13.4 tok/s | Q4_K_M · 17.9 GB | |
| 5 | LFM2.5 24B A2BLiquid AI Q4_K_M · 15.0 GB · 115 t/s | 115 tok/s | Q4_K_M · 15.0 GB |
| # | Model | Speed | Reasoning Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 16.5 GB · 74.7 t/s | 74.7 tok/s | Q4_K_M · 16.5 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 36.8 t/s | 36.8 tok/s | Q4_K_M · 6.6 GB | |
| 3 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 80.7 t/s | 80.7 tok/s | Q4_K_M · 13.1 GB | |
| 4 | Magistral Small 2506Mistral AI Q4_K_M · 17.8 GB · 15.7 t/s | 15.7 tok/s | Q4_K_M · 17.8 GB | |
| 5 | LFM2.5 24B A2BLiquid AI Q4_K_M · 15.0 GB · 115 t/s | 115 tok/s | Q4_K_M · 15.0 GB |
| # | Model | Speed | Long context Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 36.8 t/s | 36.8 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 71.8 t/s | 71.8 tok/s | Q4_K_M · 3.4 GB | |
| 3 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 80.7 t/s | 80.7 tok/s | Q4_K_M · 13.1 GB | |
| 4 | Mistral Nemo Instruct 2407Mistral AI Q4_K_M · 11.0 GB · 29.0 t/s | 29.0 tok/s | Q4_K_M · 11.0 GB | |
| 5 | LFM2.5 24B A2BLiquid AI Q4_K_M · 15.0 GB · 115 t/s | 115 tok/s | Q4_K_M · 15.0 GB |
| # | Model | Speed ↓ | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | GPT OSS 20BOpenAIHow to run → Q4_K_M · 13.1 GB · 80.7 t/s | 80.7 tok/s | Q4_K_M · 13.1 GB | |
| 2 | LFM2.5 24B A2BLiquid AI Q4_K_M · 15.0 GB · 115 t/s | 115 tok/s | Q4_K_M · 15.0 GB | |
| 3 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 74.7 t/s | 74.7 tok/s | Q4_K_M · 16.5 GB | |
| 4 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 71.8 t/s | 71.8 tok/s | Q4_K_M · 3.4 GB | |
| 5 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 36.8 t/s | 36.8 tok/s | Q4_K_M · 6.6 GB |
| # | Model | Speed | Vision Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 16.5 GB · 74.7 t/s | 74.7 tok/s | Q4_K_M · 16.5 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 36.8 t/s | 36.8 tok/s | Q4_K_M · 6.6 GB | |
| 3 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 71.8 t/s | 71.8 tok/s | Q4_K_M · 3.4 GB | |
| 4 | Gemma 3 12B ITGoogle Q4_K_M · 10.9 GB · 29.1 t/s | 29.1 tok/s | Q4_K_M · 10.9 GB | |
| 5 | Q4_K_M · 6.5 GB · 37.7 t/s | 37.7 tok/s | Q4_K_M · 6.5 GB |
The RTX 4000 Ada Generation runs 14B models well. Ada Lovelace generation, released 2023.
RTX 4000 Ada Generation strengths and weaknesses
Strengths
- Runs 35B MoE models: Qwen3.6 35B A3B at 75.1 tok/s in 19.8 GB
- Reads long prompts fast: A 4K-token prompt on Gemma 4 26B A4B IT starts answering in 749 ms
- Low power draw: 130 W board power; a 300 W PSU is enough
Weaknesses
- 70B models don't fit: Llama 3.3 70B Instruct needs 51.3 GB at Q4 vs 20 GB here
- Slow on large dense models: Qwen3.8 27B at 13.4 tok/s
Output speed depends on memory bandwidth (360 GB/s). Grey rows need more than 20 GB.
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB36.8tok/s
- 14BQwen3 14B · Q4_K_M · 11.9 GB24.6tok/s
- 27–31BQwen3.8 27B · Q4_K_M · 17.9 GB13.4tok/s
- 75.1tok/s
- 70BLlama 3.3 70B Instruct · not at 4-bitneeds 51.3 GB
- 120B MoEGPT OSS 120B · not at 4-bitneeds 70.8 GB
Estimated speeds: Q4_K_M, 4K context, one user. How we estimate
All 1117 models on the RTX 4000 Ada Generation
Featured onlyBest quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
Holo4 35B A3BVisionFunctions | IQ4_XS | 19.3 GB | 88.2 t/s | 262K | B62, Decent | |
| Q4_K_M | 0.3 GB | 650.0 t/s | 1K | S100, Runs great | ||
Laguna XS 2.1ChatFunctions | Q4_K_S | 20 GB | 104.1 t/s | 262K | C44, Usable | |
Qwen3 0.6BChatFunctions | Q4_K_M | 1.7 GB | 269.0 t/s | 41K | S100, Runs great | |
Emo 1b14b 1TChat | Q4_K_M | 9.0 GB | 201.4 t/s | 4K | S100, Runs great | |
North Mini Code 1.0ChatCodeFunctions | Q4_K_M | 19.4 GB | 88.1 t/s | 500K | B62, Decent | |
ClefVisionFunctions | Q4_K_M | 17.6 GB | 13.6 t/s | 262K | C42, Usable | |
| Q4_K_M | 1.5 GB | 207.1 t/s | 262K | S100, Runs great | ||
Llm Jp 4.1 32B A3b ThinkingChatFunctions | Q3_K_M | 16.5 GB | 100.8 t/s | 66K | S88, Runs great | |
InternVL3 5 1BVision | Q4_K_M | 2.2 GB | 334.3 t/s | — | S100, Runs great | |
| Q4_K_M | 0.9 GB | 329.6 t/s | 262K | S100, Runs great | ||
MiniCPM5 1B SFTChat | Q4_K_M | 1.3 GB | 236.4 t/s | 131K | S100, Runs great | |
Deeplm 108MChat | BF16 | 1.0 GB | 975.0 t/s | — | S100, Runs great | |
K2 Horizon 7BChatFunctions | Q4_K_M | 8.1 GB | 39.0 t/s | 524K | A75, Runs well | |
GPT OSS 20B RichardErkhov HeresyChatFunctions | Q4_K_M | 13.1 GB | 80.7 t/s | 131K | S88, Runs great |
Rent NVIDIA RTX 4000 Ada Generation in the cloud
Hourly rate on demand for one RTX 4000 Ada Generation.
RunPod$5+ bonus creditLowest price
from$0.20/hr
Vast.aiMarketplace
from$0.21/hr
Buying one ($1,250) costs as much as ~6,300 hours of rental, about 26 months at 8 hours a day.
Cost per million tokens
Output-token cost on an RTX 4000 Ada Generation rented at $0.20/hr, one request at a time. Batch several users on the same card and the cost per token drops several-fold.
| Model | Quant | Speed | Per 1M output |
|---|---|---|---|
| LFM2.5 24B A2B | 115 tok/s | $0.48$0.0057 input | |
| NVIDIA Nemotron 3.5 Lightning 30B A3B BF16 | 111 tok/s | $0.50$0.0055 input | |
| GPT OSS 20B | 81 tok/s | $0.69$0.0097 input | |
| Gemma 4 26B A4B IT | 75 tok/s | $0.74$0.010 input | |
| Qwen3.8 27B | 13 tok/s | $4.15$0.064 input |
Specs
- Architecture
- Ada Lovelace
- VRAM
- 20 GB GDDR6
- Memory bandwidth
- 360.0 GB/s
- FP16 (dense)
- 106.90 TFLOPS
- CUDA cores
- 6,144
- Tensor cores
- 192
- Compute capability
- 8.9
- TDP
- 130 W
- Released
- 2023-08-09
- MSRP
- $1,250
RTX 4000 Ada Generation FAQ
What is the largest model the RTX 4000 Ada Generation can run?
Qwen3.6 35B A3B (36.0B) is the largest at 4-bit or better: at IQ4_XS it uses 19.8 GB of 20 GB, and runs at about 75.1 tok/s.
Can the RTX 4000 Ada Generation run Gemma 4 26B A4B IT?
Yes. Gemma 4 26B A4B IT fits at Q4_K_M, using 16.5 GB of 20 GB, and it is the best model that runs well on this card.
Can the RTX 4000 Ada Generation run 70B models?
No. Llama 3.3 70B Instruct needs 51.3 GB at Q4, more than the 20 GB on this card.
How many tokens per second does the RTX 4000 Ada Generation give?
About 74.7 tok/s on Gemma 4 26B A4B IT at Q4_K_M, with the first token after 0.7 s on a 4K-token prompt.
What power supply does the RTX 4000 Ada Generation need?
Plan a 300 W PSU (2× the 130 W TDP).
Is 20 GB enough for coding and agents?
Yes for coding: Gemma 4 26B A4B IT, the best of the 3 coding models that fit, runs at about 75 tok/s. For agents, not yet: no agent-scored model fits in less than 96 GB.