Best LLMs for 24 GB VRAM
Enthusiast (RTX 3090, RTX 4090, RX 7900 XTX) — 30B+ models at Q4-Q6, 70B at aggressive quant
24 GB is the enthusiast tier for running AI models locally. It comfortably handles 7B–13B models at high quality and opens the door to larger 30B models at moderate quantization.
This is one of the most popular memory tiers for local AI, found in GPUs like the RTX 4090 and RTX 3090. You can run Llama 3 8B, Mistral 7B, and Qwen 2.5 7B at Q5_K_M or Q6_K quality with fast token generation and generous context windows. Larger 14B models like DeepSeek R1 Distill fit comfortably at Q4_K_M. For even bigger models, 30B class runs at Q2–Q3, but 70B models are generally too heavy for single-GPU inference at this tier.
Runs Well
- 7B models (Llama 3 8B, Mistral 7B) at Q5–Q8 quality
- 13B–14B models at Q4–Q5 quality
- Small models (3B–4B) at FP16 precision
- Multimodal models like LLaVA 7B
Challenging
- 30B models only at Q2–Q3 quantization
- 70B models do not fit in VRAM
- Large context windows with 14B+ models
GPUs with ~24.0 GB VRAM
All 13 GPUsAMD Radeon RX 7900 XTX
AMD · RDNA 3
NVIDIA RTX PRO 4000 Blackwell
NVIDIA · Blackwell
NVIDIA RTX PRO 4000 Blackwell SFF Edition
NVIDIA · Blackwell
NVIDIA Tesla M40 24GB
NVIDIA · Maxwell
NVIDIA L4
NVIDIA · Ada Lovelace
Intel Arc Pro B60
Intel · Xe2 (Battlemage), BMG-G21
Models That Fit in 24 GB VRAM
Speed estimated for NVIDIA GeForce RTX 4090
247 models · 178 excellent · 35 good
| Model | Quant | VRAM | Speed | Context | Status | Grade |
|---|---|---|---|---|---|---|
Q4_K_M·183.0 t/s tok/s·262K ctx·RUNS GREAT | Q4_K_M | 16.1 GB | 183.0 t/s | 262K | RUNS GREAT | S99 |
Q4_K_M·37.6 t/s tok/s·262K ctx·RUNS WELL | Q4_K_M | 17.4 GB | 37.6 t/s | 262K | RUNS WELL | A74 |
Q4_K_M·35.7 t/s tok/s·131K ctx·RUNS WELL | Q4_K_M | 18.3 GB | 35.7 t/s | 131K | RUNS WELL | A73 |
Q4_K_M·183.0 t/s tok/s·262K ctx·RUNS GREAT | Q4_K_M | 16.1 GB | 183.0 t/s | 262K | RUNS GREAT | S99 |
Q4_K_M·176.6 t/s tok/s·262K ctx·RUNS GREAT | Q4_K_M | 19.3 GB | 176.6 t/s | 262K | RUNS GREAT | S97 |
Q4_K_M·172.8 t/s tok/s·500K ctx·RUNS GREAT | Q4_K_M | 18.7 GB | 172.8 t/s | 500K | RUNS GREAT | S98 |
Q4_K_M·141.8 t/s tok/s·203K ctx·RUNS GREAT | Q4_K_M | 19.8 GB | 141.8 t/s | 203K | RUNS GREAT | S92 |
Q4_K_M·171.3 t/s tok/s·262K ctx·RUNS GREAT | Q4_K_M | 21.4 GB | 171.3 t/s | 262K | RUNS GREAT | S86 |
Q4_K_M·173.1 t/s tok/s·262K ctx·RUNS GREAT | Q4_K_M | 18.7 GB | 173.1 t/s | 262K | RUNS GREAT | S98 |
Q4_K_M·184.4 t/s tok/s·262K ctx·RUNS GREAT | Q4_K_M | 21.2 GB | 184.4 t/s | 262K | RUNS GREAT | S88 |
Q4_K_M·103.0 t/s tok/s·262K ctx·RUNS GREAT | Q4_K_M | 6.4 GB | 103.0 t/s | 262K | RUNS GREAT | S92 |
Q4_K_M·32.1 t/s tok/s·262K ctx·RUNS WELL | Q4_K_M | 20.4 GB | 32.1 t/s | 262K | RUNS WELL | A65 |
Q4_K_M·257.3 t/s tok/s·131K ctx·RUNS GREAT | Q4_K_M | 7.7 GB | 257.3 t/s | 131K | RUNS GREAT | S100 |
Q4_K_M·79.6 t/s tok/s·262K ctx·RUNS GREAT | Q4_K_M | 8.2 GB | 79.6 t/s | 262K | RUNS GREAT | S88 |
Q4_K_M·420.9 t/s tok/s·128K ctx·RUNS GREAT | Q4_K_M | 5.5 GB | 420.9 t/s | 128K | RUNS GREAT | S100 |
Q4_K_M·151.0 t/s tok/s·262K ctx·RUNS WELL | Q4_K_M | 21.9 GB | 151.0 t/s | 262K | RUNS WELL | A81 |
Frequently Asked Questions
- What models can I run with 24.0 GB VRAM?
With 24.0 GB VRAM, you can run 2313 LLM models at various quantization levels. Popular models that fit well include Gemma 4 26B A4B IT, Qwen3.8 27B, Muse Glimmer 30B. 1888 models achieve excellent performance at this VRAM level. At this tier, you have the flexibility to choose higher quantizations (Q5/Q6) for better quality on smaller models, or run larger models at Q4.
- Is 24.0 GB enough for local AI?
24.0 GB is excellent for local AI. You have access to 2313 compatible models, from small 7B assistants to large 30B+ parameter models. This is the enthusiast tier where most popular open-source LLMs work well out of the box. You can run coding assistants, chat models, and reasoning models without worrying about VRAM limits.
- What GPU should I get for 24.0 GB VRAM?
Popular GPUs with ~24.0 GB include AMD Radeon RX 7900 XTX, NVIDIA RTX PRO 4000 Blackwell, NVIDIA RTX PRO 4000 Blackwell SFF Edition. The NVIDIA GeForce RTX 3090 Ti leads in memory bandwidth at 1008.0 GB/s, which translates directly to faster token generation. When choosing a GPU for AI, memory bandwidth matters as much as VRAM capacity — it determines how fast the model can generate text. A newer GPU with the same VRAM but higher bandwidth will produce tokens significantly faster.
Higher memory bandwidth = faster token generation. All these GPUs have approximately 24 GB VRAM, but speed varies significantly by bandwidth.
Memory bandwidth comparison
1008 GB/s1008 GB/s960 GB/s936.2 GB/s896 GB/s- How to choose the right model size for 24.0 GB?
The key rule: your model must fit in VRAM including KV cache overhead. With 24.0 GB, here's a practical guide: 7B models at Q6–Q8 give you the best quality output. 14B models at Q4–Q5 offer a great quality/size balance. 30B+ models fit at Q4 but leave less room for context. Start with a 7B model at high quality and scale up as needed.
- Should I get 24.0 GB or 48.0 GB for AI?
Upgrading from 24.0 GB to 48.0 GB gives you significantly more flexibility. At 24.0 GB you can run 2313 models; with 48.0 GB you'll unlock larger models and higher-quality quantizations. If budget allows, the extra VRAM is always worth it for AI workloads — you can't add VRAM later.