AMD · RDNA 2
Best AI Models for AMD Radeon RX 6700M (10 GB)
Runs 7–9B models well; 14B only just fits.
10 GB GDDR6320 GB/s21 TFLOPS FP16135 WRDNA 2 2021
10 GB at 320 GB/s fits ~13B Q4 models with a short context. Not on AMD's official ROCm list; the Vulkan backend (llama.cpp, LM Studio) is the easy path.
| # | Model | Speed | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 30.2 t/s | 30.2 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 58.9 t/s | 58.9 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Gemma 2 9B ITGoogle Q4_K_M · 7.3 GB · 31.5 t/s | 31.5 tok/s | Q4_K_M · 7.3 GB | |
| 4 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 67.6 t/s | 67.6 tok/s | Q4_K_M · 5.0 GB | |
| 5 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 103 t/s | 103 tok/s | Q4_K_M · 5.5 GB |
| # | Model | Speed | Coding Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Q4_K_M · 7.9 GB · 36.2 t/s | 36.2 tok/s | Q4_K_M · 7.9 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 30.2 t/s | 30.2 tok/s | overall | Q4_K_M · 6.6 GB |
| 3 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 58.9 t/s | 58.9 tok/s | overall | Q4_K_M · 3.4 GB |
| 4 | Gemma 4 E4B ITGoogle Q4_K_M · 5.6 GB · 36.1 t/s | 36.1 tok/s | Q4_K_M · 5.6 GB | |
| 5 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 103 t/s | 103 tok/s | Q4_K_M · 5.5 GB |
| # | Model | Speed | Reasoning Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 30.2 t/s | 30.2 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3 8BAlibaba Q4_K_M · 7.6 GB · 34.8 t/s | 34.8 tok/s | Q4_K_M · 7.6 GB | |
| 3 | Q4_K_M · 7.9 GB · 36.2 t/s | 36.2 tok/s | Q4_K_M · 7.9 GB | |
| 4 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 67.6 t/s | 67.6 tok/s | Q4_K_M · 5.0 GB | |
| 5 | Gemma 4 E4B ITGoogle Q4_K_M · 5.6 GB · 36.1 t/s | 36.1 tok/s | Q4_K_M · 5.6 GB |
| # | Model | Speed | Long context Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 30.2 t/s | 30.2 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 58.9 t/s | 58.9 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Q4_K_M · 7.9 GB · 36.2 t/s | 36.2 tok/s | Q4_K_M · 7.9 GB | |
| 4 | Gemma 4 E4B ITGoogle Q4_K_M · 5.6 GB · 36.1 t/s | 36.1 tok/s | Q4_K_M · 5.6 GB | |
| 5 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 103 t/s | 103 tok/s | Q4_K_M · 5.5 GB |
| # | Model | Speed ↓ | llmrun Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 58.9 t/s | 58.9 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 103 t/s | 103 tok/s | Q4_K_M · 5.5 GB | |
| 3 | Gemma 4 E4B ITGoogle Q4_K_M · 5.6 GB · 36.1 t/s | 36.1 tok/s | Q4_K_M · 5.6 GB | |
| 4 | Qwen3 8BAlibaba Q4_K_M · 7.6 GB · 34.8 t/s | 34.8 tok/s | Q4_K_M · 7.6 GB | |
| 5 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 30.2 t/s | 30.2 tok/s | Q4_K_M · 6.6 GB |
| # | Model | Speed | Vision Score | Quant · VRAM |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 30.2 t/s | 30.2 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 58.9 t/s | 58.9 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 67.6 t/s | 67.6 tok/s | Q4_K_M · 5.0 GB | |
| 4 | Q4_K_M · 6.5 GB · 30.9 t/s | 30.9 tok/s | Q4_K_M · 6.5 GB | |
| 5 | MiniCPM V 4.7 1BOpenBMB Q4_K_M · 1.6 GB · 150 t/s | 150 tok/s | Q4_K_M · 1.6 GB |
See all 246 models on the RX 6700M →Best quant that fits · speeds estimated on this card
The RX 6700M runs 7–9B models well. RDNA 2 generation, released 2021.
GenerationRDNA 2released 2021
RX 6700M strengths and weaknesses
Strengths
- Runs 14B models: Qwen3 14B at 26.4 tok/s in 9.6 GB
- Low power draw: 135 W board power; a 300 W PSU is enough
Weaknesses
- 27–31B models don't fit: Qwen3.8 27B needs 17.9 GB at Q4 vs 10 GB here
- Little room for context: Qwen3 14B uses 9.6 of 10 GB, 0.4 GB left
- Slow prompt reading: A 4K-token prompt on Qwen3.5 9B waits 16.9 s for the first token
- ROCm, not CUDA: No FP8 weights; MXFP4 only through llama.cpp, not vLLM
Cards people weigh against the RX 6700M, each with the one reason it is worth a look. Speeds on Qwen3.5 9B.
2 GB more and 22% fasterRTX 3060 12GBNVIDIA · 12 GB · Ampere
▲ 22% speed▲ 2 GBabout $329rent $0.06/hr
2 GB more and 71% fasterRTX 4070NVIDIA · 12 GB · Ada Lovelace▲ 71% speed▲ 2 GBabout $599rent $0.12/hr
2 GB more and 35% fasterRX 9070 GREAMD · 12 GB · RDNA 4▲ 35% speed▲ 2 GBabout $549
2 GB more and 20% fasterRX 6800MAMD · 12 GB · RDNA 2▲ 20% speed▲ 2 GB
Best value near the RX 6700MRTX 5070NVIDIA · 12 GB · Blackwell▲ 127% speed▲ 2 GBabout $840rent $0.14/hr
6 GB more and 2.5× the speedRTX 4080 SUPERNVIDIA · 16 GB · Ada Lovelace▲ 149% speed▲ 6 GBabout $999rent $0.23/hr
Output speed depends on memory bandwidth (320 GB/s). Grey rows need more than 10 GB.
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB30.2tok/s
- 26.4tok/s
- 27–31BQwen3.8 27B · not at 4-bitneeds 17.9 GB
- 35B MoEQwen3.6 35B A3B · not at 4-bitneeds 22.0 GB
- 70BLlama 3.3 70B Instruct · not at 4-bitneeds 51.3 GB
- 120B MoEGPT OSS 120B · not at 4-bitneeds 70.8 GB
Estimated speeds: Q4_K_M, 4K context, one user. How we estimate
All 923 models on the RX 6700M
Featured onlyBest quant that fits, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
Lucie 7BChat | Q4_K_M | 6.5 GB | 41.8 t/s | 32K | A76, Runs well | |
Gpt2 XlChat | Q4_K_M | 1.1 GB | 181.1 t/s | 1K | S99, Runs great | |
TinyLLama V0Chat | Q4_K_M | 0.0 GB | 19200.0 t/s | 2K | S100, Runs great | |
Yi 1.5 6B ChatChat | Q4_K_M | 4.2 GB | 47.2 t/s | 4K | A78, Runs well | |
| Q4_K_M | 1.9 GB | 104.3 t/s | 8K | S92, Runs great | ||
| Q4_K_M | 0.0 GB | 19200.0 t/s | 1K | S100, Runs great | ||
Pythia 1.4BChat | Q4_K_M | 1 GB | 192.0 t/s | 2K | S99, Runs great | |
Shieldgemma 9BChat | Q4_K_M | 8.8 GB | 31.5 t/s | — | B61, Decent | |
Gemma 3n E2B ITVision | Q4_K_M | 5.9 GB | 53.5 t/s | — | A81, Runs well | |
Gemma 4 E2BChat | Q4_K_M | 3.5 GB | 56.0 t/s | 131K | A82, Runs well | |
| Q4_K_M | 7.6 GB | 34.8 t/s | 33K | A73, Runs well | ||
| Q4_K_M | 6.2 GB | 35.6 t/s | 8K | A73, Runs well | ||
Mamba 2.8B HFChat | Q4_K_M | 3.7 GB | 104.9 t/s | — | S92, Runs great | |
Llama 68MChat | Q4_K_M | 0.0 GB | 4800.0 t/s | 2K | S100, Runs great | |
Granite 3.1 3B A800m InstructChatFunctions | Q4_K_M | 3.4 GB | 102.5 t/s | 131K | S92, Runs great |
Specs
- Architecture
- RDNA 2
- VRAM
- 10 GB GDDR6
- Memory bandwidth
- 320.0 GB/s
- FP16 (dense)
- 21.20 TFLOPS
- Stream processors
- 2,304
- TDP
- 135 W
- Released
- 2021-05-31
RX 6700M FAQ
What is the largest model the RX 6700M can run?
Qwen3.5 9B (9.7B) is the largest at 4-bit or better: at Q4_K_M it uses 6.6 GB of 10 GB, and runs at about 30.2 tok/s.
Can the RX 6700M run 27–31B models?
No. Qwen3.8 27B needs 17.9 GB at Q4, more than the 10 GB on this card.
How many tokens per second does the RX 6700M give?
About 30.2 tok/s on Qwen3.5 9B at Q4_K_M, with the first token after 17.0 s on a 4K-token prompt.
What power supply does the RX 6700M need?
Plan a 300 W PSU (2× the 135 W TDP).
Is 10 GB enough for coding and agents?
Yes for coding: Llama 3.1 8B Instruct, the only coding-scored model that fits, runs at about 36 tok/s. For agents, not yet: no agent-scored model fits in less than 96 GB.
Is renting cheaper than buying an RX 6700M?
An RTX 3080 Ti (12 GB) rents from $0.13/hr. With regular daily use, owning pays off.