llmrun

NVIDIA · Turing · 24 GB VRAM tier

Best AI Models for NVIDIA TITAN RTX (24 GB)

Runs 27–31B and 35B MoE models well; 70B doesn't fit in 24 GB.

24 GB GDDR6672 GB/s65 TFLOPS FP16280 WTuring 2018
Buyabout $2,499

Turing 24 GB prosumer card with Tensor Cores; cheaper used than a 3090 but slower and no FP8/BF16.

What LLMs can the TITAN RTX run?

#ModelSpeedllmrun ScoreQuant · VRAM
1
Qwen3.8 27BAlibabaHow to run →
Q4_K_M · 17.9 GB · 25.1 t/s
25.1 tok/sQ4_K_M · 17.9 GB
2
Qwen3.6 27BAlibaba
Q4_K_M · 17.9 GB · 25.1 t/s
25.1 tok/sQ4_K_M · 17.9 GB
3
Gemma 4 31B ITGoogle
Q4_K_M · 21.7 GB · 21.4 t/s
21.4 tok/sQ4_K_M · 21.7 GB
4
Gemma 4 26B A4B ITGoogle
Q4_K_M · 16.5 GB · 126 t/s
126 tok/sQ4_K_M · 16.5 GB
5
NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA
Q4_K_M · 19.3 GB · 162 t/s
162 tok/sQ4_K_M · 19.3 GB
See all 327 models on the TITAN RTX →Best quant that fits · speeds estimated on this card

About the TITAN RTX

The TITAN RTX is an entry card in the 24 GB class: it runs 27–31B and 35B MoE models well. It is more efficient per watt than most 24 GB cards, and at its $2,499 MSRP it cost more per token than most of the cards compared. Turing generation, released 2018.

Speed in 24 GB#5672 GB/s
Per watt#30.25 tok/s per W on Qwen3.5 9B (9B params)
Per dollar#82.7 tok/s per $100 on Qwen3.5 9B (9B params)
GenerationTuringreleased 2018

TITAN RTX strengths and weaknesses

Strengths

  • Runs 35B MoE models: Qwen3.6 35B A3B at 113 tok/s in 22 GB
  • Room for long context: Qwen3.8 27B at Q4 uses 17.9 GB, leaving 6.1 GB free

Weaknesses

  • 70B models don't fit: Llama 3.3 70B Instruct needs 51.3 GB at Q4 vs 24 GB here
  • Slow prompt reading: A 4K-token prompt on Qwen3.8 27B waits 7.7 s for the first token
  • No FP8 or NVFP4: Turing predates both; use GGUF, AWQ or GPTQ builds

Speed per dollar and per watt

On Qwen3.5 9B (9B params) at Q4_K_M. New prices are compared with new, used with used. The last two columns are the speed you get per $100 of price and per 100 W of board power; higher is better. The TITAN RTX has no current price, so it is shown at its MSRP.

CardNew pricetok/stok/s per $100tok/s per 100 W
RX 7900 XTX24 GB
about $1,13090.6
8.0
25.5
RTX 5070 Ti16 GBBest new value
about $1,17091.6
7.8
30.5
RX 907016 GB
about $80060.4
7.6
27.5
RX 9070 XT16 GB
about $81560.4
7.4
19.9
RTX 508016 GB
about $1,69098.1
5.8
27.3
RX 9060 XT 16GB16 GB
about $59030.2
5.1
18.9
RTX 509032 GB
about $5,376183
3.4
31.8
TITAN RTX24 GB
about $2,49968.7
2.7
24.5
CardUsed pricetok/stok/s per $100tok/s per 100 W
RX 907016 GBBest used value
about $65060.4
9.3
27.5
RX 7900 XTX24 GB
about $1,00090.6
9.1
25.5
RTX 5070 Ti16 GB
about $1,10091.6
8.3
30.5
RX 9070 XT16 GB
about $79060.4
7.6
19.9
RTX 4070 Ti SUPER16 GB
about $90068.7
7.6
24.1
RTX 408016 GB
about $1,10073.3
6.7
22.9
RTX 4080 SUPER16 GB
about $1,15575.2
6.5
23.5
RTX 509032 GB
about $6,400183
2.9
31.8

Cost per token: the TITAN RTX against 16–32 GB cards

On Qwen3.5 9B (9B params, Q4_K_M, 4K context). Speed, cost per token and power per token depend on the model.

16 GB24 GB32 GBthis cardat MSRP
$0.00$0.50$1.00$1.50$2.00$2.50$3.00$0$2,000$4,000$6,000$ per 1M output tokens · lower is betternew price →RTX 5090 · 32 GB · ≈$5,376 · $1.99 per 1M tokensRX 7900 XTX · 24 GB · ≈$1,130 · $0.95 per 1M tokensRTX 5070 Ti · 16 GB · ≈$1,170 · $0.95 per 1M tokensRX 9070 · 16 GB · ≈$800 · $0.99 per 1M tokensRTX 5080 · 16 GB · ≈$1,690 · $1.25 per 1M tokensRX 9070 XT · 16 GB · ≈$815 · $1.07 per 1M tokensRX 9060 XT 16GB · 16 GB · ≈$590 · $1.46 per 1M tokensTITAN RTX · 24 GB · $2,499 MSRP · $2.48 per 1M tokensTITAN RTXRTX 5070 TiRX 9070RTX 5090RTX 5080RX 9070 XTRX 9060 XT 16GB
$0.00$0.50$1.00$1.50$2.00$2.50$3.00$0$2k$4k$6k$ per 1M output tokens · lower is betternew price →RTX 5090 · 32 GB · ≈$5,376 · $1.99 per 1M tokensRX 7900 XTX · 24 GB · ≈$1,130 · $0.95 per 1M tokensRTX 5070 Ti · 16 GB · ≈$1,170 · $0.95 per 1M tokensRX 9070 · 16 GB · ≈$800 · $0.99 per 1M tokensRTX 5080 · 16 GB · ≈$1,690 · $1.25 per 1M tokensRX 9070 XT · 16 GB · ≈$815 · $1.07 per 1M tokensRX 9060 XT 16GB · 16 GB · ≈$590 · $1.46 per 1M tokensTITAN RTX · 24 GB · $2,499 MSRP · $2.48 per 1M tokensTITAN RTXRTX 5070 TiRX 9070
Cost per 1M output tokenson Qwen3.5 9B · card over 3 years + power
$2.48
Card, MSRP
$2.31
Power
$0.17
3-year cost of ownership$2,683
Or rent in the cloud$0.22/hr

Renting costs as much as buying after ≈11,600 hours (≈7.9 years at 4 h a day).

Include card price

Each dot is a card; hover or tap it for its numbers, click to open it. Cards without a price for the chosen basis are left out. How we calculate cost per token

Similar GPUs

Cards people weigh against the TITAN RTX, each with the one reason it is worth a look. Speeds on Qwen3.5 9B.

32% faster, same 24 GBRX 7900 XTXAMD · 24 GB · RDNA 3
▲ 32% speedsame 24 GBabout $1,130
8 GB more and 2.7× the speedRTX 5090NVIDIA · 32 GB · Blackwell
▲ 167% speed▲ 8 GBabout $5,376rent $0.50/hr
9% faster, 16 GBRTX 4080 SUPERNVIDIA · 16 GB · Ada Lovelace
▲ 9% speed▼ 8 GBabout $999rent $0.24/hr
10% faster, 20 GBRX 7900 XTAMD · 20 GB · RDNA 3
▲ 10% speed▼ 4 GBabout $899
More efficient per wattRTX 5090 Laptop GPUNVIDIA · 24 GB · Blackwell
▲ 33% speedsame 24 GB
Best value near the TITAN RTXRTX 5070 TiNVIDIA · 16 GB · Blackwell
▲ 33% speed▼ 8 GBabout $1,170rent $0.17/hr

TITAN RTX tokens per second by model size

Output speed depends on memory bandwidth (672 GB/s). Grey rows need more than 24 GB.

  1. 9BQwen3.5 9B · Q4_K_M · 6.6 GB
    68.7tok/s
  2. 14BQwen3 14B · Q4_K_M · 11.9 GB
    46.0tok/s
  3. 27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
    25.1tok/s
  4. 35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
    113tok/s
  5. 70BLlama 3.3 70B Instruct · not at 4-bit
    needs 51.3 GB
  6. 120B MoEGPT OSS 120B · not at 4-bit
    needs 70.8 GB

Estimated speeds: Q4_K_M, 4K context, one user. How we estimate

All 1141 models on the TITAN RTX

Featured only

Best quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works

LLM models compatible with NVIDIA TITAN RTX — ranked by Score, recency and Fit
ModelVRAMScoreFit
Kimi K2.6 Eagle3
Q4_K_M·299.2 t/s
1.9 GBS100, Runs great
GaMS3 12B Instruct
Q4_K_M·53.9 t/s
8.9 GBA81, Runs well
Kappa 20B 131k Mxfp4
Q4_K_M·137.9 t/s
13.1 GBS96, Runs great
QwQ 32B
Q4_K_S·22.7 t/s
23.0 GBC38, Usable
Carbon 8B
Q4_K_M·79.1 t/s
7.4 GBS87, Runs great
Supra Mini V5 8M
Q4_K_M·1365.0 t/s
0.3 GBS100, Runs great
Soren 1 Small
BF16·105.0 t/s
4.3 GBS92, Runs great
Granite Switch 4.1 3B Preview
BF16·49.8 t/s
10.0 GBA79, Runs well
AI21 Jamba Reasoning 3B
Q4_K_M·194.1 t/s
2.3 GBS99, Runs great
SmolVLM2 256M Video Instruct
Q4_K_M·873.6 t/s
0.6 GBS100, Runs great
Tiny Aya Global
BF16·59.3 t/s
9.4 GBA84, Runs well
Gemma 3 12B IT Heretic v2
Q4_K_M·52.2 t/s
9.2 GBA81, Runs well
GPT X2 125M
BF16·682.5 t/s
0.6 GBS100, Runs great
Deepseek Vl2 Tiny
BF16·116.8 t/s
9.4 GBS94, Runs great
Vllm Translategemma 12B IT
Q4_K_M·48.7 t/s
9.8 GBA79, Runs well

TITAN RTX specs, power and PSU

Specs

Architecture
Turing
VRAM
24 GB GDDR6
Memory bandwidth
672.0 GB/s
FP16 (dense)
65.25 TFLOPS
CUDA cores
4,608
Tensor cores
576
Compute capability
7.5
TDP
280 W
Released
2018-12-18
MSRP
$2,499

TITAN RTX FAQ

Is the TITAN RTX still worth it in 2026?

At its MSRP of $2,499 you get 2.7 tok/s per $100 on Qwen3.5 9B, which ranks 8th of the 8 cards compared here; it is also the 5th fastest out of 5 cards with 24 GB. It runs 27–31B and 35B MoE models well; 70B doesn't fit in 24 GB.

What is the largest model the TITAN RTX can run?

Qwen3.6 35B A3B (36.0B) is the largest at 4-bit or better: at Q4_K_M it uses 22.0 GB of 24 GB, and runs at about 113 tok/s.

Can the TITAN RTX run Qwen3.8 27B?

Yes. Qwen3.8 27B fits at Q4_K_M, using 17.9 GB of 24 GB, and it is the best model that runs well on this card.

Can the TITAN RTX run 70B models?

No. Llama 3.3 70B Instruct needs 51.3 GB at Q4, more than the 24 GB on this card.

How many tokens per second does the TITAN RTX give?

About 25.1 tok/s on Qwen3.8 27B at Q4_K_M, with the first token after 7.8 s on a 4K-token prompt.

What power supply does the TITAN RTX need?

Plan a 600 W PSU (2× the 280 W TDP).

Is 24 GB enough for coding and agents?

Yes for coding: Qwen3.8 27B, the best of the 9 coding models that fit, runs at about 25 tok/s. For agents, not yet: no agent-scored model fits in less than 96 GB.

Is renting cheaper than buying a TITAN RTX?

Rent from $0.22/hr. Renting costs as much as buying after ≈11,600 hours (≈7.9 years at 4 h a day).

Related

Best coding LLMs24 GB VRAM48 GB VRAMGPUs with 24 GB or moreRTX 3090 TiRTX 3090