llmrun
Thinking MachinesInklingInklingForConditionalGeneration

Inkling — Hardware Requirements & GPU Compatibility

VisionChatFunctions

Inkling is the flagship model in Thinking Machines' Inkling family, a vision-language chat model with roughly 952 billion parameters. It accepts image input alongside text and supports general conversational use, and was released about two weeks ahead of the smaller Inkling Small model in the same line, in mid-July 2026. Inkling is distributed under the Apache 2.0 license, allowing free local and commercial use without restriction. Given its scale, it needs multi-GPU or server-class hardware to run at all; most users will access it through a hosted endpoint rather than running it on their own machines.

952.4B params · MoE18B activeApache 2.01.3M downloads1.8K likes396.1K quant downloads

Inkling runs on a multi-GPU server

Q4_K_M needs ~575.1 GB · about 8× H100 SXM 80 GB · or Mac Studio (M3 Ultra, 512GB) at Q3_K_M (~35 tok/s)

Macs from512 GB Q3_K_M · ~35 tok/s

Score 116#18 of 100 open#15 of 96 that fit ≤640 GBApache 2.0

Not set up for it?

Reference GPU · RTX 4090 24 GB

Estimates, ±20% · how we estimate

llmrun Composite Score →

116.0

Benchmarks · tier chart ↓
Among 100 open modelsHow the Composite Score is built

Run Inkling

One recommended way. Every other method, OS and quant is in the step-by-step guide.

Recommended

vLLM on a multi-GPU server
tensor + pipeline parallel · needs vLLM on every node
$vllm serve thinkingmachines/Inkling --max-model-len 16384 --tensor-parallel-size 8

Size the nodes from the table above

Download: inkling-Q4_K_M.gguf571.43 GBunsloth

install, other ways

Other ways

each opens step-by-step

Run in cloud

Rent 8× H100 SXM node · Q4_K_M

Estimated hourly price
~$21.52/hr
est., whole server
GPUs
8 GPUs
640 GB of VRAM
Deployon RunPodRent H100 SXM$2.69/hr per GPU (opens in new tab)
Compare GPUs →

How Much VRAM Does Inkling Need?

Select a quantization level; the listed hardware adheres to it.

QuantizationVRAM+32KRuns on
405.5 GB411.7 GB2-bit · K-quant — 2-bit quantization with K-quant improvementsRuns on 6× H100 SXM 80 GB
465 GB471.2 GB3-bit medium — 3-bit medium quantizationRuns on 7× H100 SXM 80 GB
572.1 GB578.4 GB4-bit medium — 4-bit medium quantization — most popular sweet spotRuns on 8× H100 SXM 80 GB
679.3 GB685.5 GB5-bit medium — 5-bit medium quantization — good quality/size tradeoffRuns on 6× H200 SXM 141 GB
786.4 GB792.7 GB6-bit, very good — 6-bit quantization, very good qualityRuns on 7× H200 SXM 141 GB
953.1 GB959.3 GB8-bit, near-lossless — 8-bit quantization, near-losslessRuns on 8× H200 SXM 141 GB
1905.5 GB1911.7 GB16-bit · bfloat — Brain floating point 16 — preferred for trainingRuns on 4 nodes of 8× H100 SXM
est.calculated, no published file yet

Selected Q4_K_M

4-bit medium quantization

575.1GBat 16K context

Weights
571.4 GB
KV cache
3.3 GB
Overhead
0.3 GB

Server-class: ~575.1 GB at 16K context · about 8× H100 SXM 80 GB

Which Hardware Can Run Inkling?

Q4_K_M · 572.1 GB

Inkling (Q4_K_M) requires 572.1 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 744+ GB is recommended. No consumer GPU has enough memory.

Needs server-class hardware

Inkling runs on a multi-GPU server · Q4_K_M needs ~575.1 GB

Hardware that runs this model
HardwareQuantMemorySpeedContextHardware page
Mac 512 GB · Mac Studio (M3 Ultra, 512GB)Q3_K_M467.9 GB~35 tok/s32K ctxMac Studio →Q3_K_M · 467.9 GB · ~35 tok/s · 32K ctx
8× H100 SXM 80 GB · 640 GBQ4_K_M575.1 GB—16K ctxH100 SXM →Q4_K_M · 575.1 GB · 16K ctx
5× H200 SXM 141 GB · 705 GBQ4_K_M575.1 GB—32K ctxH200 SXM →Q4_K_M · 575.1 GB · 32K ctx
4× B200 192 GB · 768 GBQ4_K_M575.1 GB—32K ctxB200 →Q4_K_M · 575.1 GB · 32K ctx

Also: NVIDIA DGX A100 640GB · NVIDIA DGX H100

≤ 640 GB tier →

By VRAM tier · typical GPUs and Macs at Q4_K_M

Consumer GPUs (up to 32 GB) and Macs up to 256 GB: too small for this model (needs ≥ 408.4 GB)8 GB tier →12 GB tier →16 GB tier →24 GB tier →Macs →

How good is Inkling?

The llmrun Score merges 25 public benchmark boards into a single figure, where 100 represents the average for open models.

Score vs VRAM in the ≤640 GB tier

  • Inkling
  • Other models
  • Best Score for its VRAM

#15 of 96 that fit ≤640 GB

Each dot is a model at the best quant that fits ≤640 GB. The line joins the models with the best Score for their VRAM; a dot below it is beaten by a model that needs no more memory.

Full ≤640 GB ranking →

Benchmarks

The 6 boards that weigh most in the llmrun Score, of 31 with a result for this model; badges show who ran the test.

All 31 Inkling benchmarks

In the ≤640 GB class

96 models fit ≤640 GB

Same Score, only models that fit ≤640 GB at their best quant: this model's peers by memory need.

  1. #1

    MiMo V2.6 Pro RLXiaomi

    133.4

    614.1 GB

  2. #2

    MiMo V2.6 Flash RLXiaomi

    131.9

    186.5 GB

  3. #3

    GLM 5.3Z.ai

    130.1

    453.8 GB

  4. #4

    DeepSeek V4.1 FlashDeepSeek

    129.5

    458.4 GB

  5. #5

    DeepSeek V4 Flash 0731DeepSeek

    127.6

    183.0 GB

  6. #6

    Muse Spark Open 816B A42BMeta

    125.3

    551.9 GB

Full ≤640 GB ranking →

Alternatives to Inkling

Inkling family · 3 models →

Official models with an llmrun Score, each answering one question; sizes are each model's default quant (usually Q4_K_M) with a 16K-token context.

Similar models

10
ModelScoreVRAMWhy it is similar
DeepSeek V4 Pro 0813DeepSeek129.2990.8 GBsimilar score
Muse Spark Open 816B A42BMeta125.3551.9 GBsimilar score
GLM 5.2Z.ai122.9453.8 GBsimilar score
Qwen3.8 27BAlibaba122.117.9 GBsimilar score
Kimi K2.6Moonshot AI120.9617.6 GBsimilar score
GLM 5.1Z.ai117.8454.1 GBsimilar score
DeepSeek V4 ProDeepSeek116.5959.8 GBsimilar score
MiniMax M3MiniMax112.5258 GBsimilar score
Qwen3.6 27BAlibaba111.217.9 GBsimilar score
DeepSeek V3.2DeepSeek110.4412.7 GBsimilar score

Specifications

Publisher
Thinking Machines
Family
Inkling
Parameters
952.4B
Active parameters
18.1B per token
Architecture
InklingForConditionalGeneration
Vocabulary Size
201,024
Release Date
2026-07-14
License
Apache 2.0

Resources

Frequently Asked Questions

How much VRAM does Inkling need?

Inkling requires 572.1 GB of VRAM at Q4_K_M, or 1905.5 GB at BF16.

VRAM = Weights + KV Cache + Overhead

Weights = 952.4B × 4.8 bits ÷ 8 = 571.4 GB

KV Cache + Overhead ≈ 0.7 GB (at 2K context + ~0.3 GB framework)

Fit ratings and hardware model lists check this model with room for a 16K-token context, which needs a little more memory.

VRAM usage by quantization

572.1 GB

Learn more about VRAM estimation →

Can NVIDIA GeForce RTX 5090 run Inkling?

No — Inkling requires at least 405.5 GB at Q2_K, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.

What's the best quantization for Inkling?

For Inkling, Q4_K_M (572.1 GB) offers the best balance of quality and VRAM usage. Q5_K_M (679.3 GB) provides better quality if you have the VRAM. The smallest option is Q2_K at 405.5 GB.

VRAM requirement by quantization

Q2_K
405.5 GB
Q4_K_M ★
572.1 GB
Q5_K_M
679.3 GB
Q6_K
786.4 GB
Q8_0
953.1 GB
BF16
1905.5 GB

★ Recommended — best balance of quality and VRAM usage.

Learn more about quantization →

Can I run Inkling on a Mac?

Yes, but only at lower quantizations. The smallest Mac that can run Inkling is Mac Studio (M3 Ultra, 512GB) at Q3_K_M; 1 of the 146 Macs we list can run it at some quantization. For Q4_K_M (572.1 GB) you need a Mac with more unified memory.

Can I run Inkling locally?

Yes — Inkling can run locally on consumer hardware. Popular tools include Ollama, LM Studio, and llama.cpp.

What's the download size of Inkling?

At Q4_K_M, the download is about 571.43 GB. The full-precision BF16 version is 1904.76 GB. The smallest option (Q2_K) is 404.76 GB.

Which GPUs can run Inkling?

No single consumer GPU has enough VRAM to run Inkling at Q4_K_M (572.1 GB). Multi-GPU or professional hardware is required.

Which devices can run Inkling?

Inkling requires at least 572.1 GB at Q4_K_M, which exceeds the usable memory of every unified-memory device we list (the largest has 508.5 GB).