NVIDIA · Maxwell · 24 GB VRAM tier
Best AI Models for NVIDIA Tesla M40 24GB
Runs 14B and 35B MoE models well; 27–31B fits but is slow (11 tok/s).
24 GB GDDR5288 GB/s7 TFLOPS FP16250 WMaxwell 2015
Maxwell (2015): no Tensor Cores, slow FP16, GDDR5 — the cheapest 24 GB card but also the slowest here. Passive server card; needs added cooling.
| # | Model | Speed | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 56.6 t/s | 56.6 tok/s | Q4_K_M · 22.0 GB | |
| 2 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 61.3 t/s | 61.3 tok/s | Q4_K_M · 16.5 GB | |
| 3 | Qwen3 30B A3B Thinking 2507Alibaba Q4_K_M · 19.4 GB · 73.4 t/s | 73.4 tok/s | Q4_K_M · 19.4 GB | |
| 4 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 66.0 t/s | 66.0 tok/s | Q4_K_M · 13.1 GB | |
| 5 | Q4_K_M · 19.3 GB · 94.5 t/s | 94.5 tok/s | Q4_K_M · 19.3 GB |
| # | Model | Speed | Coding Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 56.6 t/s | 56.6 tok/s | Q4_K_M · 22.0 GB | |
| 2 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 61.3 t/s | 61.3 tok/s | Q4_K_M · 16.5 GB | |
| 3 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 66.0 t/s | 66.0 tok/s | Q4_K_M · 13.1 GB | |
| 4 | Q4_K_M · 7.9 GB · 35.3 t/s | 35.3 tok/s | Q4_K_M · 7.9 GB | |
| 5 | Q4_K_M · 19.3 GB · 94.5 t/s | 94.5 tok/s | Q4_K_M · 19.3 GB |
| # | Model | Speed | Reasoning Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 56.6 t/s | 56.6 tok/s | Q4_K_M · 22.0 GB | |
| 2 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 61.3 t/s | 61.3 tok/s | Q4_K_M · 16.5 GB | |
| 3 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 29.4 t/s | 29.4 tok/s | Q4_K_M · 6.6 GB | |
| 4 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 66.0 t/s | 66.0 tok/s | Q4_K_M · 13.1 GB | |
| 5 | Q4_K_M · 19.3 GB · 94.5 t/s | 94.5 tok/s | Q4_K_M · 19.3 GB |
| # | Model | Speed | Math Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 56.6 t/s | 56.6 tok/s | Q4_K_M · 22.0 GB | |
| 2 | Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 10.7 t/s | 10.7 tok/s | Q4_K_M · 17.9 GB | |
| 3 | Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 10.7 t/s | 10.7 tok/s | Q4_K_M · 17.9 GB | |
| 4 | Q4_K_M · 19.3 GB · 94.5 t/s | 94.5 tok/s | Q4_K_M · 19.3 GB | |
| 5 | GLM 4.7 FlashZ.ai Q4_K_M · 19.9 GB · 54.9 t/s | 54.9 tok/s | Q4_K_M · 19.9 GB |
| # | Model | Speed | Long context Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 16.5 GB · 61.3 t/s | 61.3 tok/s | Q4_K_M · 16.5 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 29.4 t/s | 29.4 tok/s | Q4_K_M · 6.6 GB | |
| 3 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 57.4 t/s | 57.4 tok/s | Q4_K_M · 3.4 GB | |
| 4 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 66.0 t/s | 66.0 tok/s | Q4_K_M · 13.1 GB | |
| 5 | LFM2.5 24B A2BLiquid AI Q4_K_M · 15.0 GB · 96.9 t/s | 96.9 tok/s | Q4_K_M · 15.0 GB |
| # | Model | Speed ↓ | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 19.4 GB · 73.4 t/s | 73.4 tok/s | Q4_K_M · 19.4 GB | |
| 2 | Q4_K_M · 19.3 GB · 94.5 t/s | 94.5 tok/s | Q4_K_M · 19.3 GB | |
| 3 | Qwen3 30B A3B Instruct 2507Alibaba Q4_K_M · 19.4 GB · 73.4 t/s | 73.4 tok/s | Q4_K_M · 19.4 GB | |
| 4 | GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 66.0 t/s | 66.0 tok/s | Q4_K_M · 13.1 GB | |
| 5 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 61.3 t/s | 61.3 tok/s | Q4_K_M · 16.5 GB |
| # | Model | Speed | Vision Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 56.6 t/s | 56.6 tok/s | Q4_K_M · 22.0 GB | |
| 2 | Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 61.3 t/s | 61.3 tok/s | Q4_K_M · 16.5 GB | |
| 3 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 29.4 t/s | 29.4 tok/s | Q4_K_M · 6.6 GB | |
| 4 | Gemma 3 12B ITGoogle Q4_K_M · 10.9 GB · 23.3 t/s | 23.3 tok/s | Q4_K_M · 10.9 GB | |
| 5 | MiniCPM V 4.7 35B A3BOpenBMB Q4_K_M · 22.1 GB · 65.5 t/s | 65.5 tok/s | Q4_K_M · 22.1 GB |
See all 327 models on the Tesla M40 24GB →Best quant that fits · speeds estimated on this card
The Tesla M40 24GB runs 14B and 35B MoE models well. Maxwell generation, released 2015.
GenerationMaxwellreleased 2015
Tesla M40 24GB strengths and weaknesses
Strengths
- Runs 35B MoE models: Qwen3.6 35B A3B at 56.6 tok/s in 22 GB
Weaknesses
- 70B models don't fit: Llama 3.3 70B Instruct needs 51.3 GB at Q4 vs 24 GB here
- Slow on large dense models: Qwen3.8 27B at 10.7 tok/s
- Slow prompt reading: A 4K-token prompt on Qwen3.6 35B A3B waits 12.6 s for the first token
Output speed depends on memory bandwidth (288 GB/s). Grey rows need more than 24 GB.
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB29.4tok/s
- 14BQwen3 14B · Q4_K_M · 11.9 GB19.7tok/s
- 27–31BQwen3.8 27B · Q4_K_M · 17.9 GB10.7tok/s
- 35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB56.6tok/s
- 70BLlama 3.3 70B Instruct · not at 4-bitneeds 51.3 GB
- 120B MoEGPT OSS 120B · not at 4-bitneeds 70.8 GB
Estimated speeds: Q4_K_M, 4K context, one user. How we estimate
All 1141 models on the Tesla M40 24GB
Featured onlyBest quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
Qwen1.5 32B ChatChat | Q4_K_S | 22.9 GB | 9.8 t/s | 33K | D25, Barely runs | |
Qwen1.5 1.8BChat | Q4_K_M | 4.6 GB | 104.0 t/s | 33K | S92, Runs great | |
Qwen2.5 Coder 7BChatCodeFunctions | Q4_K_M | 5.8 GB | 37.5 t/s | 33K | A74, Runs well | |
OpenReasoning Nemotron 32BChatCodeReasoning | Q3_K_M | 20.6 GB | 11.1 t/s | 131K | C42, Usable | |
Llama 2 7B HFChat | Q4_K_M | 6.9 GB | 42.1 t/s | — | A76, Runs well | |
Aya 23 8BChat | BF16 | 20.2 GB | 10.6 t/s | — | C43, Usable | |
Yi 34BChat | Q4_K_M | 21.9 GB | 8.7 t/s | 4K | C30, Usable | |
| Q4_K_M | 7.9 GB | 33.8 t/s | 131K | A73, Runs well | ||
Internlm Chat 7BChat | FP16 | 15.4 GB | 12.2 t/s | 2K | B51, Decent | |
Llama 2 13B HFChat | Q4_K_M | 11.5 GB | 21.8 t/s | — | B62, Decent | |
| IQ3_M | 23.5 GB | 29.8 t/s | 33K | C43, Usable | ||
Yi 34B ChatChat | Q4_K_M | 21.9 GB | 8.7 t/s | 4K | C30, Usable | |
QwQ 32B PreviewChatReasoning | Q4_K_S | 23.0 GB | 9.7 t/s | 33K | D23, Barely runs | |
Qwen2.5 Coder 14BChatCode | Q4_K_M | 12.4 GB | 19.6 t/s | 33K | B60, Decent | |
Gemma 7B ITChat | BF16 | 21.4 GB | 10.0 t/s | — | C36, Usable |
Specs
- Architecture
- Maxwell
- VRAM
- 24 GB GDDR5
- Memory bandwidth
- 288.0 GB/s
- FP16 (dense)
- 6.83 TFLOPS
- CUDA cores
- 3,072
- Tensor cores
- 0
- Compute capability
- 5.2
- TDP
- 250 W
- Released
- 2015-11-10
Tesla M40 24GB FAQ
What is the largest model the Tesla M40 24GB can run?
Qwen3.6 35B A3B (36.0B) is the largest at 4-bit or better: at Q4_K_M it uses 22.0 GB of 24 GB, and runs at about 56.6 tok/s.
Can the Tesla M40 24GB run 70B models?
No. Llama 3.3 70B Instruct needs 51.3 GB at Q4, more than the 24 GB on this card.
How many tokens per second does the Tesla M40 24GB give?
About 56.6 tok/s on Qwen3.6 35B A3B at Q4_K_M, with the first token after 12.7 s on a 4K-token prompt.
What power supply does the Tesla M40 24GB need?
Plan a 500 W PSU (2× the 250 W TDP).
Is 24 GB enough for coding and agents?
Yes for coding: Qwen3.6 35B A3B, the best of the 4 coding models that fit, runs at about 57 tok/s. For agents, not yet: no agent-scored model fits in less than 96 GB.