NVIDIA · Jetson T5000 (Thor) · AI box

Best AI Models for NVIDIA Jetson AGX Thor Developer Kit (128 GB)

Runs 14B and 120B MoE models well; 27–31B and 70B fit but are slow (10 and 4 tok/s).

128 GB unified112 GB for models273 GB/s259 TFLOPS FP16130 W2025

What LLMs can the NVIDIA Jetson AGX Thor Developer Kit run?

#	Model	Speed	Quant · Memory
1	Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 54.0 t/s	54.0 tok/s	Q4_K_M · 22.0 GB
2	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 58.4 t/s	58.4 tok/s	Q4_K_M · 16.5 GB
3	Qwen3 30B A3B Thinking 2507Alibaba Q4_K_M · 19.4 GB · 70.3 t/s	70.3 tok/s	Q4_K_M · 19.4 GB
4	GLM 4.5 AirZ.ai Q4_K_M · 67.6 GB · 18.7 t/s	18.7 tok/s	Q4_K_M · 67.6 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 90.9 t/s	90.9 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed	Quant · Memory
1	Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 54.0 t/s	54.0 tok/s	Q4_K_M · 22.0 GB
2	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 58.4 t/s	58.4 tok/s	Q4_K_M · 16.5 GB
3	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 62.8 t/s	62.8 tok/s	Q4_K_M · 13.1 GB
4	GPT OSS 120BOpenAI Q4_K_M · 70.8 GB · 46.4 t/s	46.4 tok/s	Q4_K_M · 70.8 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 90.9 t/s	90.9 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed	Quant · Memory
1	Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 54.0 t/s	54.0 tok/s	Q4_K_M · 22.0 GB
2	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 58.4 t/s	58.4 tok/s	Q4_K_M · 16.5 GB
3	Qwen3.5 9BAlibaba Q4_K_M · 6.6 GB · 27.9 t/s	27.9 tok/s	Q4_K_M · 6.6 GB
4	GPT OSS 120BOpenAI Q4_K_M · 70.8 GB · 46.4 t/s	46.4 tok/s	Q4_K_M · 70.8 GB
5	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 90.9 t/s	90.9 tok/s	Q4_K_M · 19.3 GB

#	Model	Speed	Quant · Memory
1	Qwen3.6 35B A3BAlibabaHow to run → Q4_K_M · 22.0 GB · 54.0 t/s	54.0 tok/s	Q4_K_M · 22.0 GB
2	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 10.2 t/s	10.2 tok/s	Q4_K_M · 17.9 GB
3	Qwen3.6 27BAlibaba Q4_K_M · 17.9 GB · 10.2 t/s	10.2 tok/s	Q4_K_M · 17.9 GB
4	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 90.9 t/s	90.9 tok/s	Q4_K_M · 19.3 GB
5	GLM 4.7 FlashZ.ai Q4_K_M · 19.9 GB · 52.4 t/s	52.4 tok/s	Q4_K_M · 19.9 GB

#	Model	Speed ↓	Quant · Memory
1	Qwen3 30B A3B Thinking 2507AlibabaHow to run → Q4_K_M · 19.4 GB · 70.3 t/s	70.3 tok/s	Q4_K_M · 19.4 GB
2	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 90.9 t/s	90.9 tok/s	Q4_K_M · 19.3 GB
3	Qwen3 30B A3B Instruct 2507Alibaba Q4_K_M · 19.4 GB · 70.3 t/s	70.3 tok/s	Q4_K_M · 19.4 GB
4	GPT OSS 20BOpenAI Q4_K_M · 13.1 GB · 62.8 t/s	62.8 tok/s	Q4_K_M · 13.1 GB
5	Gemma 4 26B A4B ITGoogle Q4_K_M · 16.5 GB · 58.4 t/s	58.4 tok/s	Q4_K_M · 16.5 GB

#	Model	Speed	Agents Score	Quant · Memory
1	GPT OSS 120BOpenAIHow to run → Q4_K_M · 70.8 GB · 46.4 t/s	46.4 tok/s		Q4_K_M · 70.8 GB
2	Qwen3.8 27BAlibaba Q4_K_M · 17.9 GB · 10.2 t/s	10.2 tok/s		Q4_K_M · 17.9 GB
3	Qwen3.6 35B A3BAlibaba Q4_K_M · 22.0 GB · 54.0 t/s	54.0 tok/s	overall	Q4_K_M · 22.0 GB
4	NVIDIA Nemotron 3.5 Lightning 30B A3B BF16NVIDIA Q4_K_M · 19.3 GB · 90.9 t/s	90.9 tok/s		Q4_K_M · 19.3 GB
5	GLM 4.7 FlashZ.ai Q4_K_M · 19.9 GB · 52.4 t/s	52.4 tok/s		Q4_K_M · 19.9 GB

See all 399 models on the NVIDIA Jetson AGX Thor Developer Kit →Bars = traction for models without a Score yet

About the NVIDIA Jetson AGX Thor Developer Kit

The NVIDIA Jetson AGX Thor Developer Kit is a 128 GB AI box with more memory than speed: it runs 14B and 120B MoE models well. The system keeps about 16 GB, leaving 112 GB for models. With 259 TFLOPS it reads long prompts faster than most 128 GB machines: a 4K-token prompt takes 330 ms. At its $3,499 MSRP it gives 15.4 tok/s per $1,000, 4th of the 8 machines compared here. Jetson T5000 (Thor) (Blackwell), released 2025.

Speed in 128 GB#6273 GB/s

Prompt reading#1259 TFLOPS

For models112 GBThe system keeps 16 GB

ChipJetson T5000 (Thor)released 2025

NVIDIA Jetson AGX Thor Developer Kit strengths and weaknesses

Strengths

Runs 120B MoE models: GPT OSS 120B at 46.4 tok/s in 70.8 GB
Reads long prompts fast: A 32K-token prompt on Qwen3.6 35B A3B starts answering in 2.7 s
Low power draw: 130 W at full load for the whole machine
FP8 and NVFP4 run natively: Blackwell takes FP8 and NVFP4 checkpoints in vLLM as published

Weaknesses

Slow on large dense models: Llama 3.3 70B Instruct at 3.8 tok/s, Qwen3.8 27B at 10.2 tok/s

Memory for models

112 GB of 128 GB

35B MoE22 GB70B51.3 GB120B MoE70.8 GB127B82.8 GB

35B MoE22 GB127B82.8 GB

For models 112 GBThe system keeps 16 GBModel size at Q4

Everything on the size ladder fits. Qwen3.6 35B A3B and Gemma 4 26B A4B IT together use 38.5 GB and leave 73.5 GB free.

Speed per dollar and per watt

NewUsed

On Qwen3.6 35B A3B (35B params, 3B active) at Q4_K_M. New means a shop price, either a street quote or the maker's own price; used is compared with used. The last two columns are the speed you get per $1,000 of price and per 100 W of power; higher is better. The NVIDIA Jetson AGX Thor Developer Kit has no current new price, so it is shown at its MSRP.

Machine	New price	tok/s	tok/s per $1,000	tok/s per 100 W
RTX 509032 GB	about $5,376	213	39.7	37.1
Framework Desktop Ryzen AI Max+ 395128 GB	about $1,999	46.7	23.4	33.4
Mac Studio M4 Max128 GB	about $3,499	64.2	18.3	44.3
Jetson AGX Thor Developer Kit128 GB	about $3,499	54.0	15.4	41.5
Mac Studio M2 Ultra192 GB	about $5,599	75.9	13.6	25.7
DGX Spark128 GB	about $3,999	54.0	13.5	22.5
MacBook Pro 16" M5 Max128 GB	about $5,399	67.8	12.6	–
Mac Studio M3 Ultra256 GB	about $7,099	76.6	10.8	28.4

Machine	Used price	tok/s	tok/s per $1,000	tok/s per 100 W
RTX 409024 GB	about $3,000	151	50.3	33.6
RTX 509032 GB	about $6,400	213	33.3	37.1

Cost per token: the NVIDIA Jetson AGX Thor Developer Kit against 24–256 GB machines and GPUs

On Qwen3.6 35B A3B (35B params, 3B active, Q4_K_M, 4K context). Speed, cost per token and power per token depend on the model.

24–36 GB96–128 GB192+ GBthis AI boxat MSRP

Cost per 1M output tokenson Qwen3.6 35B A3B · card over 3 years + power

$4.21

Machine, MSRP: $4.11
Power: $0.10

3-year cost of ownership$3,584

Or rent an A100 in the cloud$1.00/hr

Renting costs as much as buying after ≈3,500 hours (≈2.4 years at 4 h a day).

Include card price

Power$/kWhUseh/dayKeepyears

Each dot is a machine or GPU; hover or tap it for its numbers, click to open it. Machines without a price for the chosen basis are left out. Machines without a published power figure are left out of Per watt and costed without power. How we calculate cost per token

Similar devices and alternatives

The same chip from other makers first, then machines and GPUs people weigh against it, each with the one reason it is worth a look. Speeds on Qwen3.6 35B A3B.

Machine, as fastDGX SparkNVIDIA · 120 GB for models · GB10

same speed▲ 8 GB$3,999

19% faster, 12.5 GB more memoryMac Studio M4 Max (128 GB)Apple · 124.5 GB for models · M4 Max

▲ 19% speed▲ 12.5 GB$3,499

26% faster, 12.5 GB more memoryMacBook Pro 16" M5 Max (128 GB)Apple · 124.5 GB for models · M5 Max

▲ 26% speed▲ 12.5 GB$5,399

Machine, 14% slowerFramework Desktop Ryzen AI Max+ 395Framework · 96 GB for models · Ryzen AI Max+ 395

▼ 13% speed▼ 16 GB$1,999

2.8× faster if your models fit in 24 GBRTX 4090NVIDIA · 24 GB VRAM · GPU

▲ 180% speed▼ 88 GB$1,599rent $0.34/hr

4.0× faster if your models fit in 32 GBRTX 5090NVIDIA · 32 GB VRAM · GPU

▲ 295% speed▼ 80 GB$5,376rent $0.50/hr

NVIDIA Jetson AGX Thor Developer Kit tokens per second by model size

Output speed depends on memory bandwidth (273 GB/s). Grey rows need more than the 112 GB available.

9BQwen3.5 9B · Q4_K_M · 6.6 GB
27.9tok/s
14BQwen3 14B · Q4_K_M · 11.9 GB
18.7tok/s
27–31BQwen3.8 27B · Q4_K_M · 17.9 GB
10.2tok/s
35B MoEQwen3.6 35B A3B · Q4_K_M · 22.0 GB
54.0tok/s
70BLlama 3.3 70B Instruct · Q4_K_M · 51.3 GB slow
3.8tok/s
120B MoEGPT OSS 120B · Q4_K_M · 70.8 GB
46.4tok/s

Estimated speeds: Q4_K_M, 4K context, one user. How we estimate

Unified-memory platforms: writing vs compute

Writing, tok/sCompute, TFLOPS

Orin64 GB · 204.8 GB/s
41.7
43
Ryzen AI Max+ 395128 GB · 256 GB/s
46.7
30
GB10128 GB · 273 GB/s
54.0
125
Jetson T5000 (Thor)128 GB · 273 GB/s
54.0
259
M4 Pro64 GB · 273 GB/s
43.2
17

Machines with about the same memory bandwidth write at about the same speed; compute decides how fast prompts are read. Speeds on Qwen3.6 35B A3B.

All 399 models on the NVIDIA Jetson AGX Thor Developer Kit

Featured only

Best quant that fits in 112 GB, ordered by llmrun Score. Fit grades how well it runs here, S best. How ranking works

LLM models compatible with NVIDIA Jetson AGX Thor Developer Kit — ranked by Score, recency and Fit
Model	Quant	VRAM	Speed	Context	Fit
Ornith 1.5 122B125.1BChatFunctions Q4_K_M·21.8 t/s	Q4_K_M	75.5 GB67%	21.8 t/s	262K	B62, Decent
JEV 27B VL27.8BVisionFunctions Q4_K_M·10.2 t/s	Q4_K_M	17.9 GB16%	10.2 t/s	262K	B48, Decent
Mistral Medium 3.5 128B127.7BChatFunctions Q4_K_M·2.3 t/s	Q4_K_M	82.8 GB74%	2.3 t/s	262K	C38, Usable
MiniMax M2.7228.7BChatFunctions IQ3_M·31.7 t/s	IQ3_M	105.3 GB94%	31.7 t/s	205K	B50, Decent
Ornith 1.5 35B A3B36.0BChatFunctions Q4_K_M·54.0 t/s	Q4_K_M	22.0 GB20%	54.0 t/s	262K	A81, Runs well
MiniCPM5 2B2.5BChatFunctions Q4_K_M·93.4 t/s	Q4_K_M	2.5 GB2%	93.4 t/s	131K	S90, Runs great
Hy Memory 30B A3B30.1BChatFunctions Q4_K_M·76.9 t/s	Q4_K_M	19.1 GB17%	76.9 t/s	262K	S87, Runs great
MiMo V2.6 Distill Qwen 9B9.4BVisionCodeFunctions Q4_K_M·28.6 t/s	Q4_K_M	6.5 GB6%	28.6 t/s	262K	A69, Runs well
Qwen3.5 4B4.7BVisionFunctions Q4_K_M·54.4 t/s	Q4_K_M	3.4 GB3%	54.4 t/s	262K	A81, Runs well
Spark X2.6 4B4.1BChatFunctions Q4_K_M·59.9 t/s	Q4_K_M	4.3 GB4%	59.9 t/s	1049K	A84, Runs well
Ornith 1.0 35B35.1BChatFunctions Q4_K_M·64.0 t/s	Q4_K_M	21.5 GB19%	64.0 t/s	262K	A84, Runs well
Qwen3.5 9B9.7BVisionFunctions Q4_K_M·27.9 t/s	Q4_K_M	6.6 GB6%	27.9 t/s	262K	A69, Runs well
ZAYA1 VL 8B9.7BVision Q4_K_M·210.3 t/s	Q4_K_M	7.5 GB7%	210.3 t/s	33K	S100, Runs great
GLM 4.6V107.7BVisionFunctions Q4_K_M·19.6 t/s	Q4_K_M	66.0 GB59%	19.6 t/s	131K	B60, Decent
MiniCPM V 4.7 1B1.6BVisionFunctions Q4_K_M·138.6 t/s	Q4_K_M	1.6 GB1%	138.6 t/s	262K	S96, Runs great

NVIDIA Jetson AGX Thor Developer Kit specs and power

Specs

Chip: Jetson T5000 (Thor) · Blackwell GPU (Jetson Thor)
CPU cores: 14
GPU cores: 2,560
Unified memory: 128 GB LPDDR5X
For models: 112 GB (The system keeps 16 GB)
Memory bandwidth: 273 GB/s
FP16 (dense): 258.8 TFLOPS
Power: 130 W max
Released: 2025-08-25
MSRP: $3,499

NVIDIA Jetson AGX Thor Developer Kit FAQ

Is the NVIDIA Jetson AGX Thor Developer Kit worth it for local LLMs in 2026?

At its $3,499 MSRP it gives 15.4 tok/s per $1,000 on Qwen3.6 35B A3B, 4th of the 8 machines compared here. It runs 14B and 120B MoE models well; 27–31B and 70B fit but are slow (10 and 4 tok/s).

How much memory is available for AI on the NVIDIA Jetson AGX Thor Developer Kit?

About 112 GB of the 128 GB. The system keeps roughly 16 GB for itself and other apps.

What is the largest model the NVIDIA Jetson AGX Thor Developer Kit can run?

Mistral Medium 3.5 128B (127.7B) is the largest at 4-bit or better: at Q4_K_M it uses 82.8 GB, 74% of the available memory, and runs at about 2.3 tok/s. At 3-bit, a bigger model fits too: MiniMax M2.7 (228.7B) at IQ3_M, 105.3 GB.

Can the NVIDIA Jetson AGX Thor Developer Kit run Qwen3.6 35B A3B?

Yes. Qwen3.6 35B A3B fits at Q4_K_M, using 22.0 GB of the 112 GB available, and it is the best model that runs well on this machine.

Can the NVIDIA Jetson AGX Thor Developer Kit run 70B models?

Yes, but slowly: Llama 3.3 70B Instruct at Q4_K_M uses 51.3 GB and writes about 3.8 tok/s.

How long does the NVIDIA Jetson AGX Thor Developer Kit take to read a long prompt?

On Qwen3.6 35B A3B, about 330 ms for a 4K-token prompt and 2.7 s for a 32K-token document before the first word appears.

Is renting a GPU cheaper than buying the NVIDIA Jetson AGX Thor Developer Kit?

An A100 (80 GB) rents from $1.00/hr. Renting costs as much as buying after ≈3,500 hours (≈2.4 years at 4 h a day).

48 GB GPUs AI boxes Mac Pro M2 Ultra Mac Studio M1 Ultra Mac Studio M2 Ultra Mac Studio M5 Max MacBook Pro 14" M5 Max Best coding LLMs All hardware