NVIDIA · Orin Nano · AI box
Best AI Models for NVIDIA Jetson Orin Nano 8GB (Super)
Runs 9B models at 10 tok/s; 14B doesn't fit in 8 GB.
8 GB unified102 GB/s17 TFLOPS FP1625 W2023
| # | Model | Speed | llmrun Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Qwen3.5 9BAlibabaHow to run → Q4_K_M · 6.6 GB · 10.4 t/s | 10.4 tok/s | Q4_K_M · 6.6 GB | |
| 2 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 20.3 t/s | 20.3 tok/s | Q4_K_M · 3.4 GB | |
| 3 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 23.3 t/s | 23.3 tok/s | Q4_K_M · 5.0 GB | |
| 4 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 64.9 t/s | 64.9 tok/s | Q4_K_M · 5.5 GB | |
| 5 | Gemma 4 E2B ITGoogle Q4_K_M · 3.5 GB · 19.3 t/s | 19.3 tok/s | Q4_K_M · 3.5 GB |
| # | Model | Speed | Reasoning Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Q4_K_M · 5.0 GB · 23.3 t/s | 23.3 tok/s | Q4_K_M · 5.0 GB | |
| 2 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 10.4 t/s | 10.4 tok/s | Q4_K_M · 6.6 GB | |
| 3 | Qwen3.5 4BAlibaba Q4_K_M · 3.4 GB · 20.3 t/s | 20.3 tok/s | overall | Q4_K_M · 3.4 GB |
| 4 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 64.9 t/s | 64.9 tok/s | Q4_K_M · 5.5 GB | |
| 5 | Gemma 4 E2B ITGoogle Q4_K_M · 3.5 GB · 19.3 t/s | 19.3 tok/s | Q4_K_M · 3.5 GB |
| # | Model | Speed | Long context Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 20.3 t/s | 20.3 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 64.9 t/s | 64.9 tok/s | Q4_K_M · 5.5 GB | |
| 3 | Gemma 4 E2B ITGoogle Q4_K_M · 3.5 GB · 19.3 t/s | 19.3 tok/s | Q4_K_M · 3.5 GB | |
| 4 | MiniCPM5 2BOpenBMB Q4_K_M · 2.5 GB · 34.9 t/s | 34.9 tok/s | Q4_K_M · 2.5 GB | |
| 5 | Qwen3.5 2BAlibaba Q4_K_M · 1.9 GB · 37.5 t/s | 37.5 tok/s | Q4_K_M · 1.9 GB |
| # | Model | Speed ↓ | llmrun Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 20.3 t/s | 20.3 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Ling 3.0 TinyInclusion AI Q4_K_M · 5.5 GB · 64.9 t/s | 64.9 tok/s | Q4_K_M · 5.5 GB | |
| 3 | Qwen3.5 2BAlibaba Q4_K_M · 1.9 GB · 37.5 t/s | 37.5 tok/s | Q4_K_M · 1.9 GB | |
| 4 | MiniCPM5 2BOpenBMB Q4_K_M · 2.5 GB · 34.9 t/s | 34.9 tok/s | Q4_K_M · 2.5 GB | |
| 5 | Gemma 4 E2B ITGoogle Q4_K_M · 3.5 GB · 19.3 t/s | 19.3 tok/s | Q4_K_M · 3.5 GB |
| # | Model | Speed | Vision Score | Quant · Memory |
|---|---|---|---|---|
| 1 | Qwen3.5 4BAlibabaHow to run → Q4_K_M · 3.4 GB · 20.3 t/s | 20.3 tok/s | Q4_K_M · 3.4 GB | |
| 2 | Gemma 3 4B ITGoogle Q4_K_M · 5.0 GB · 23.3 t/s | 23.3 tok/s | Q4_K_M · 5.0 GB | |
| 3 | Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 10.4 t/s | 10.4 tok/s | Q4_K_M · 6.6 GB | |
| 4 | MiniCPM V 4.7 1BOpenBMB Q4_K_M · 1.6 GB · 51.8 t/s | 51.8 tok/s | Q4_K_M · 1.6 GB | |
| 5 | LFM3 Omni 2BLiquid AI Q4_K_M · 2.4 GB · 39.0 t/s | 39.0 tok/s | Q4_K_M · 2.4 GB |
See all 216 models on the NVIDIA Jetson Orin Nano 8GB (Super) →Bars = traction for models without a Score yet
The NVIDIA Jetson Orin Nano 8GB (Super) is the fastest 8 GB machine: it runs 9B models at 10 tok/s. With 17 TFLOPS it reads long prompts faster than most 8 GB machines: a 4K-token prompt takes 10 s. Orin Nano (Ampere), released 2023.
Speed in 8 GB#1102 GB/s
Prompt reading#117 TFLOPS
For models8 GBall of it
ChipOrin Nanoreleased 2023
NVIDIA Jetson Orin Nano 8GB (Super) strengths and weaknesses
Strengths
- Low power draw: 25 W at full load for the whole machine
Weaknesses
- 14B models don't fit: Qwen3 14B needs 11.9 GB at Q4 vs 8 GB usable here
- Little room for context: Qwen3.5 9B uses 6.6 of 8 GB usable, 1.4 GB left
- Slow prompt reading: A 32K-token prompt on Qwen3.5 9B waits 1.4 min for the first token
- No FP8 or NVFP4: Ampere predates both; use GGUF, AWQ or GPTQ builds
Memory for models
8 GB of 8 GB9B6.6 GB
9B6.6 GB
For models 8 GBThe system keeps 0 GBModel size at Q4
14B models need 11.9 GB at Q4, more than the 8 GB available.Or rent an RTX 3090 (24 GB) from $0.19/hr →
The same chip from other makers first, then machines and GPUs people weigh against it, each with the one reason it is worth a look. Speeds on Qwen3.5 9B.
27% faster, 4.5 GB more memoryMac Mini M4 (16 GB)Apple · 12.5 GB for models · M4
▲ 27% speed▲ 4.5 GBabout $799
189% faster, 12.5 GB more memoryMac Mini M4 Pro (24 GB)Apple · 20.5 GB for models · M4 Pro▲ 189% speed▲ 12.5 GBabout $1,599
63% faster, 20.5 GB more memoryMacBook Pro 14 M5Apple · 28.5 GB for models · M5▲ 63% speed▲ 20.5 GBabout $1,839
334% faster, 24.5 GB more memoryMac Studio M4 Max (36 GB)Apple · 32.5 GB for models · M4 Max▲ 334% speed▲ 24.5 GBabout $2,499
Reads prompts 10× faster, 24 GB for modelsRTX 4090NVIDIA · 24 GB VRAM · GPU▲ 10× prompts▲ 890% speed▲ 16 GBabout $1,599rent $0.37/hr
Reads prompts 13× faster, 32 GB for modelsRTX 5090NVIDIA · 32 GB VRAM · GPU▲ 13× prompts▲ 1661% speed▲ 24 GBabout $5,376rent $0.50/hr
Output speed depends on memory bandwidth (102 GB/s). Grey rows need more than the 8 GB available.
- 9BQwen3.5 9B · Q4_K_M · 6.6 GB10.4tok/s
- 14BQwen3 14B · not at 4-bitneeds 11.9 GB
- 27–31BQwen3.8 27B · not at 4-bitneeds 17.9 GB
- 35B MoEQwen3.6 35B A3B · not at 4-bitneeds 22.0 GB
- 70BLlama 3.3 70B Instruct · not at 4-bitneeds 51.3 GB
- 120B MoEGPT OSS 120B · not at 4-bitneeds 70.8 GB
Estimated speeds: Q4_K_M, 4K context, one user. How we estimate
Unified-memory platforms: writing vs compute
Writing, tok/sCompute, TFLOPS
- M116 GB · 68.3 GB/s7.55
- M216 GB · 100 GB/s11.07
- Orin Nano8 GB · 102 GB/s10.417
- Orin NX16 GB · 102.4 GB/s10.515
- M316 GB · 102.4 GB/s11.37
- M416 GB · 120 GB/s13.29
Machines with about the same memory bandwidth write at about the same speed; compute decides how fast prompts are read. Speeds on Qwen3.5 9B.
All 822 models on the NVIDIA Jetson Orin Nano 8GB (Super)
Featured onlyBest quant that fits in 8 GB, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works
| Model | Quant | VRAM | Speed | Context | Score | Fit |
|---|---|---|---|---|---|---|
| Q4_K_M | 7.9 GB | 12.4 t/s | — | D22, Barely runs | ||
BioMistral 7BChat | Q4_K_M | 6.7 GB | 13.9 t/s | 33K | B49, Decent | |
| Q4_K_M | 6.8 GB | 13.5 t/s | 33K | B46, Decent | ||
| Q4_K_M | 6.2 GB | 12.3 t/s | 8K | B51, Decent | ||
WildguardChat | Q4_K_M | 7.3 GB | 13.9 t/s | — | C37, Usable | |
Qwen2 1.5BChat | Q4_K_M | 1.7 GB | 51.8 t/s | 131K | A81, Runs well | |
| Q4_K_M | 7.3 GB | 12.3 t/s | 131K | C36, Usable | ||
| Q4_K_M | 7.6 GB | 12.0 t/s | 41K | D28, Barely runs | ||
Phi 4 ReasoningChatMathCodeReasoning | IQ2_XXS | 7.7 GB | 14.0 t/s | 33K | D29, Barely runs | |
Qwen2.5 Coder 3BChatCodeFunctions | Q4_K_M | 2.8 GB | 29.7 t/s | 33K | A71, Runs well | |
Qwen2.5 Omni 7BChat | Q3_K_S | 8.0 GB | 12.8 t/s | — | D22, Barely runs | |
Llama 2 7B HFChat | Q4_K_M | 6.5 GB | 12.2 t/s | 4K | B49, Decent | |
| Q4_K_M | 6.2 GB | 12.3 t/s | 8K | B51, Decent | ||
| Q4_K_M | 6.5 GB | 12.0 t/s | 4K | B48, Decent | ||
Ministral 3 14B Reasoning 2512ReasoningFunctions | IQ2_XS | 7.8 GB | 13.5 t/s | 262K | D25, Barely runs |
Specs
- Chip
- Orin Nano
- CPU cores
- 6
- GPU cores
- 1,024
- Unified memory
- 8 GB LPDDR5
- For models
- 8 GB
- Memory bandwidth
- 102 GB/s
- FP16 (dense)
- 16.8 TFLOPS
- Power
- 25 W max
- Released
- 2023-03-08
NVIDIA Jetson Orin Nano 8GB (Super) FAQ
How much memory is available for AI on the NVIDIA Jetson Orin Nano 8GB (Super)?
All 8 GB can hold model weights.
What is the largest model the NVIDIA Jetson Orin Nano 8GB (Super) can run?
Qwen3.5 9B (9.7B) is the largest at 4-bit or better: at Q4_K_M it uses 6.6 GB, 83% of the available memory, and runs at about 10.4 tok/s.
Can the NVIDIA Jetson Orin Nano 8GB (Super) run 14B models?
No. Qwen3 14B needs 11.9 GB at Q4, more than the 8 GB available.
How long does the NVIDIA Jetson Orin Nano 8GB (Super) take to read a long prompt?
On Qwen3.5 9B, about 10 s for a 4K-token prompt and 84 s for a 32K-token document before the first word appears.
Is renting a GPU cheaper than buying the NVIDIA Jetson Orin Nano 8GB (Super)?
An RTX 3080 Ti (12 GB) rents from $0.08/hr.