Run LLMs in the cloud
Live GPU prices from Vast.ai and RunPod, updated hourly. Rent a card per hour, or pick a model to find the cheapest GPU that fits it.
By model size
Weights at Q4_K_M, on the cheapest NVIDIA GPU that fits with 20% VRAM headroom. Speed is for one user at a time; cost is per output token.
| Size | VRAM at Q4 | Cheapest GPU | Speed | Per 1M output | Hourly | Deploy |
|---|---|---|---|---|---|---|
| 7–8BLlama / Qwen-class | VRAM at Q4~5.3 GB | Cheapest GPURTX 2080 Ti11 GB | Speed~76 tok/s | Per 1M output tokens$0.33 | from $0.089/hr | |
| 12–14BQwen / Gemma-class | VRAM at Q4~9.2 GB | Cheapest GPURTX 309024 GB | Speed~66 tok/s | Per 1M output tokens$0.72 | from $0.17/hr | |
| 27–32BQwen / Gemma-class | VRAM at Q4~21 GB | Cheapest GPURTX 509032 GB | Speed~55 tok/s | Per 1M output tokens$2.50 | from $0.50/hr | |
| 70BLlama / Qwen-class | VRAM at Q4~46 GB | Cheapest GPUA100 80GB | Speed~29 tok/s | Per 1M output tokens$11.51 | from $1.19/hr | |
| 100–120B MoE~12B active | VRAM at Q4~79 GB | Cheapest GPURTX PRO 600096 GB | Speed~106 tok/s | Per 1M output tokens$3.49 | from $1.33/hr | |
| 200B+235B MoE, ~22B active | VRAM at Q4~155 GB | Cheapest GPUB200192 GB | Speed~184 tok/s | Per 1M output tokens$9.02 | from $5.98/hr |
Each row uses the top of its size range. A model gets a price here when it fits on one rentable GPU (the largest right now has 288 GB); anything bigger needs several GPUs.
Popular models
The open models everyone runs most often are paired with the lowest-cost NVIDIA GPU capable of fitting them at Q4_K_M.
| Model | Params | VRAM | Cheapest GPU | Speed | Per 1M output | Hourly | Deploy |
|---|---|---|---|---|---|---|---|
| Qwen3.8 27B27.8B, ~17 GB | RTX 3090, ~35 tok/s, $1.35 per 1M output tokens | $1.35 | from $0.17/hr | ||||
| Qwen3.6 35B A3B36.0B MoE, ~22 GB | RTX A6000, ~125 tok/s, $0.73 per 1M output tokens | $0.73 | from $0.33/hr | ||||
| Gemma 4 26B A4B IT25.8B MoE, ~16 GB | RTX 3090, ~162 tok/s, $0.29 per 1M output tokens | $0.29 | from $0.17/hr | ||||
| Qwen3.5 9B9.7B, ~6.4 GB | RTX 2080 Ti, ~63 tok/s, $0.39 per 1M output tokens | $0.39 | from $0.089/hr | ||||
| Gemma 4 E4B IT8.0B, ~5.3 GB | RTX 2080 Ti, ~75 tok/s, $0.33 per 1M output tokens | $0.33 | from $0.089/hr | ||||
| Gemma 4 12B IT12.0B, ~8.2 GB | RTX 2080 Ti, ~49 tok/s, $0.51 per 1M output tokens | $0.51 | from $0.089/hr | ||||
| Qwen3.8 Flash Next180.0B MoE, ~108 GB | H200 SXM, ~69 tok/s, $14.47 per 1M output tokens | $14.47 | from $3.59/hr | ||||
| Gemma 4 31B IT31.3B, ~20 GB | RTX 5090, ~57 tok/s, $2.42 per 1M output tokens | $2.42 | from $0.50/hr | ||||
| Qwen3 8B8.2B, ~5.5 GB | RTX 2080 Ti, ~73 tok/s, $0.34 per 1M output tokens | $0.34 | from $0.089/hr | ||||
| Qwen3 4B Instruct 25074.0B, ~2.9 GB | RTX 3060 12GB, ~81 tok/s, $0.20 per 1M output tokens | $0.20 | from $0.059/hr | ||||
| Qwen2.5 7B Instruct7.6B, ~5.0 GB | RTX 2080 Ti, ~80 tok/s, $0.31 per 1M output tokens | $0.31 | from $0.089/hr | ||||
| Qwen3.5 4B4.7B, ~3.3 GB | RTX 3060 12GB, ~72 tok/s, $0.23 per 1M output tokens | $0.23 | from $0.059/hr | ||||
| Qwen3 VL 8B Instruct8.8B, ~5.9 GB | RTX 2080 Ti, ~68 tok/s, $0.36 per 1M output tokens | $0.36 | from $0.089/hr | ||||
| Qwen3.6 27B27.8B, ~17 GB | RTX 3090, ~35 tok/s, $1.35 per 1M output tokens | $1.35 | from $0.17/hr | ||||
| Gemma 4 E2B IT5.1B, ~3.4 GB | RTX 3060 12GB, ~68 tok/s, $0.24 per 1M output tokens | $0.24 | from $0.059/hr | ||||
| Qwen3 4B4.0B, ~2.9 GB | RTX 3060 12GB, ~81 tok/s, $0.20 per 1M output tokens | $0.20 | from $0.059/hr | ||||
| Qwen3 Coder 30B A3B Instruct30.5B MoE, ~19 GB | RTX 3090, ~166 tok/s, $0.28 per 1M output tokens | $0.28 | from $0.17/hr | ||||
| DeepSeek V4.1 Flash 1017763.2B MoE, ~458 GB | Requires multiple GPUs; no single rentable card will do. | ||||||
| Llama 3.2 1B Instruct1.2B, ~0.8 GB | RTX 3060 12GB, ~285 tok/s, $0.057 per 1M output tokens | $0.057 | from $0.059/hr | ||||
| Hy MT2 7B8.0B, ~5.4 GB | RTX 2080 Ti, ~74 tok/s, $0.33 per 1M output tokens | $0.33 | from $0.089/hr | ||||
| Qwen3 VL 4B Instruct4.4B, ~3.1 GB | RTX 3060 12GB, ~74 tok/s, $0.22 per 1M output tokens | $0.22 | from $0.059/hr | ||||
| GPT OSS 20B20.9B MoE, ~13 GB | RTX 3090, ~179 tok/s, $0.26 per 1M output tokens | $0.26 | from $0.17/hr | ||||
| Qwen2.5 Coder 32B Instruct32.8B, ~21 GB | RTX 5090, ~57 tok/s, $2.43 per 1M output tokens | $2.43 | from $0.50/hr | ||||
| Llama 3.1 8B Instruct8.0B, ~5.3 GB | RTX 2080 Ti, ~76 tok/s, $0.33 per 1M output tokens | $0.33 | from $0.089/hr | ||||
| Qwen2.5 VL 3B Instruct3.8B, ~2.6 GB | RTX 3060 12GB, ~89 tok/s, $0.18 per 1M output tokens | $0.18 | from $0.059/hr | ||||
| NVIDIA Nemotron 3.5 Lightning 30B A3B BF1631.6B MoE, ~19 GB | RTX 3090, ~191 tok/s, $0.25 per 1M output tokens | $0.25 | from $0.17/hr | ||||
| Qwen3 30B A3B Instruct 250730.5B MoE, ~19 GB | RTX 3090, ~166 tok/s, $0.28 per 1M output tokens | $0.28 | from $0.17/hr | ||||
| Qwen3.5 27B27.8B, ~17 GB | RTX 3090, ~35 tok/s, $1.35 per 1M output tokens | $1.35 | from $0.17/hr | ||||
| Llama 3.2 3B Instruct3.2B, ~2.1 GB | RTX 3060 12GB, ~110 tok/s, $0.15 per 1M output tokens | $0.15 | from $0.059/hr | ||||
| MiMo V2.6 Distill Qwen 9B9.4B, ~6.2 GB | RTX 2080 Ti, ~65 tok/s, $0.38 per 1M output tokens | $0.38 | from $0.089/hr | ||||
VRAM assumes Q4_K_M plus a brief context. Speed and cost apply to a single user at once, calculated per output token.
Frequently Asked Questions
What quantization should I use on a rented GPU?
Q4_K_M; every number on this page assumes it. Weights take about 4.8 bits each, so a 70B model needs roughly 46 GB including overhead instead of 140 GB at FP16, with little quality loss in chat and coding. With VRAM to spare, Q5_K_M or Q8_0 buys back a little quality for more memory and less speed. Go below Q4 only when it drops you onto a much cheaper GPU.
How long does it take to start a model on a rented GPU?
A few minutes for a small model, and up to 15 for a 70B. The machine itself is up in a minute or two; the rest is downloading weights, and a 70B at Q4 is about 40 GB, which takes a while on a host with a slow link. Billing starts when the machine boots, not when the model is ready, so stop the instance as soon as you're done.
Why is the cost per million tokens higher than an API?
The figures assume one user sending one request at a time, so most of the GPU sits idle. APIs batch many requests on the same card, so their cost per token is several times lower. Run vLLM with parallel requests and a rented GPU gets much closer. Renting still makes sense for private data, a fine-tuned model, or steady heavy use; for occasional chat an API is cheaper.