llmrun
Moonshot AIKimi K3KimiK3ForConditionalGeneration

Kimi K3 — Hardware Requirements & GPU Compatibility

VisionChat

Kimi K3 from Moonshot AI is an enormous model, weighing in at roughly 2.8 trillion parameters, among the largest open-weight releases in this catalogue. It accepts image input alongside text, supporting multimodal chat and visual reasoning in addition to standard dialogue. At this scale, local deployment is not realistic for most users; it needs multi-GPU or server-class hardware, and most people will access it through a hosted endpoint rather than running it on their own machines. Kimi K3 supports a 1M token context window, allowing it to process very long documents, codebases, or conversation histories in one pass. It was released on June 13, 2026, under its own custom license terms rather than a standard Apache or MIT license, so users should review the license text before commercial use.

2779.9B params · MoE106B active1049K contextOther1.1M downloads11.7K likes315.0K quant downloads

Kimi K3 runs on a GPU cluster

Q4_K_M needs ~1.7 TB · about 3 nodes of 8× H100 SXM 80 GB

Score 132.8#2 of 100 open#2 of 99 that fit ≤2 TBOther

Not set up for it?

Reference GPU · RTX 4090 24 GB

Estimates, ±20% · how we estimate

llmrun Composite Score →

132.8

Benchmarks · tier chart ↓
Among 100 open modelsHow the Composite Score is built

Run Kimi K3

One recommended way. Every other method, OS and quant is in the step-by-step guide.

Recommended

vLLM on GPU nodes
tensor + pipeline parallel · needs vLLM on every node
$vllm serve moonshotai/Kimi-K3 --max-model-len 16384 --tensor-parallel-size 8 --pipeline-parallel-size <nodes>

Size the nodes from the table above

Download: Kimi-K3-Q4_K_M.gguf1667.96 GBunsloth

Ray cluster setup, other ways

Other ways

each opens step-by-step

Run in cloud

Rent 3× 8-GPU H100 SXM nodes · Q4_K_M

Estimated hourly price
~$53.07/hr
est., whole cluster
GPUs
24 GPUs
1.9 TB of VRAM
Deployon Vast.aiRent H100 SXM$2.21/hr per GPU (opens in new tab)
Compare GPUs →

How Much VRAM Does Kimi K3 Need?

Select a quantization level; the listed hardware adheres to it.

QuantizationVRAM+32KRuns on
770.2 GB768.3 GB2-bit · i-quant extreme compression — Importance-weighted 2-bit, extreme compression — significant quality lossRuns on 7× H200 SXM 141 GB
1187.2 GB1185.3 GB2-bit · K-quant — 2-bit quantization with K-quant improvementsRuns on 7× B200 192 GB
1361.0 GB1359 GB3-bit medium — 3-bit medium quantizationRuns on 8× B200 192 GB
1673.7 GB1671.8 GB4-bit medium — 4-bit medium quantization — most popular sweet spotRuns on 3 nodes of 8× H100 SXM
1986.5 GB1984.5 GB5-bit medium — 5-bit medium quantization — good quality/size tradeoffRuns on 4 nodes of 8× H100 SXM
2299.2 GB2297.3 GB6-bit, very good — 6-bit quantization, very good qualityRuns on 4 nodes of 8× H100 SXM
2785.7 GB2783.7 GB8-bit, near-lossless — 8-bit quantization, near-losslessRuns on 5 nodes of 8× H100 SXM
5565.6 GB5563.7 GB16-bit · bfloat — Brain floating point 16 — preferred for trainingRuns on 10 nodes of 8× H100 SXM
est.calculated, no published file yet

Selected Q4_K_M

4-bit medium quantization

1670.0GBat 16K context

Weights
1668.0 GB
KV cache
1.8 GB
Overhead
0.3 GB

Server-class: ~1.7 TB at 16K context · about 3 nodes of 8× H100 SXM 80 GB

Which Hardware Can Run Kimi K3?

Q4_K_M · 1673.7 GB

Kimi K3 (Q4_K_M) requires 1673.7 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 2176+ GB is recommended. Using the full 1049K context window can add up to 106.9 GB, bringing total usage to 1780.6 GB. No consumer GPU has enough memory.

Needs server-class hardware

Kimi K3 runs on a GPU cluster · Q4_K_M needs ~1.7 TB

Hardware that runs this model
HardwareQuantMemoryContextHardware page
8× H200 SXM 141 GB · 1.1 TBIQ2_XXS766.5 GB1M ctxH200 SXM →IQ2_XXS · 766.5 GB · 1M ctx
8× B200 192 GB · 1.5 TBQ3_K_M1357.3 GB128K ctxB200 →Q3_K_M · 1357.3 GB · 128K ctx
GPU cluster · 3 nodes of 8× H100 SXM 80 GBQ4_K_M1670 GB512K ctxH100 SXM →Q4_K_M · 1670 GB · 512K ctx

≤ 640 GB tier →

By VRAM tier · typical GPUs and Macs at Q4_K_M

Consumer GPUs (up to 32 GB) and Macs up to 512 GB: too small for this model (needs ≥ 766.5 GB)8 GB tier →12 GB tier →16 GB tier →24 GB tier →Macs →

How good is Kimi K3?

The llmrun Score merges 30 public benchmark boards into a single figure, where 100 represents the average for open models.

Score vs VRAM in the ≤2 TB tier

  • Kimi K3
  • Other models
  • Best Score for its VRAM

#2 of 99 that fit ≤2 TB

Each dot is a model at the best quant that fits ≤2 TB. The line joins the models with the best Score for their VRAM; a dot below it is beaten by a model that needs no more memory.

Full ≤2 TB ranking →

Benchmarks

The 6 boards that weigh most in the llmrun Score, of 37 with a result for this model; badges show who ran the test.

All 37 Kimi K3 benchmarks

In the ≤2 TB class

99 models fit ≤2 TB

Same Score, only models that fit ≤2 TB at their best quant: this model's peers by memory need.

  1. #1

    MiMo V2.6 Pro RLXiaomi

    133.4

    614.1 GB

  2. #2

    Kimi K3Moonshot AI

    132.8

    1670.0 GB

  3. #3

    MiMo V2.6 Flash RLXiaomi

    131.9

    186.5 GB

  4. #4

    GLM 5.3Z.ai

    130.1

    453.8 GB

  5. #5

    DeepSeek V4.1 FlashDeepSeek

    129.5

    458.4 GB

  6. #6

    DeepSeek V4 Pro 0813DeepSeek

    129.2

    990.8 GB

Full ≤2 TB ranking →

Alternatives to Kimi K3

Official models with an llmrun Score, each answering one question; sizes are each model's default quant (usually Q4_K_M) with a 16K-token context.

Similar models

4
ModelScoreVRAMWhy it is similar
MiMo V2.6 Pro RLXiaomi133.4614.1 GBsimilar score
MiMo V2.6 Flash RLXiaomi131.9186.5 GBsimilar score
DeepSeek V4.1 FlashDeepSeek129.5458.4 GBsimilar score
DeepSeek V4 ProDeepSeek116.5959.8 GBsmaller

Specifications

Publisher
Moonshot AI
Family
Kimi K3
Parameters
2779.9B
Active parameters
106B per token
Architecture
KimiK3ForConditionalGeneration
Context Length
1,048,576 tokens
Vocabulary Size
163,840
Release Date
2026-06-13
License
Other

Resources

Frequently Asked Questions

How much VRAM does Kimi K3 need?

Kimi K3 requires 1673.7 GB of VRAM at Q4_K_M, or 5565.6 GB at BF16. Full 1049K context adds up to 106.9 GB (1780.6 GB total).

VRAM = Weights + KV Cache + Overhead

Weights = 2779.9B × 4.8 bits ÷ 8 = 1668 GB

KV Cache + Overhead ≈ 5.7 GB (at 2K context + ~0.3 GB framework)

Fit ratings and hardware model lists check this model with room for a 16K-token context, which needs a little more memory.

KV Cache + Overhead ≈ 112.6 GB (at full 1049K context)

VRAM usage by quantization

1673.7 GB
1780.6 GB

Learn more about VRAM estimation →

Can NVIDIA GeForce RTX 5090 run Kimi K3?

No — Kimi K3 requires at least 770.2 GB at IQ2_XXS, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.

What's the best quantization for Kimi K3?

For Kimi K3, Q4_K_M (1673.7 GB) offers the best balance of quality and VRAM usage. Q5_K_M (1986.5 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 770.2 GB.

VRAM requirement by quantization

IQ2_XXS
770.2 GB
Q3_K_M
1361.0 GB
Q4_K_M ★
1673.7 GB
Q5_K_M
1986.5 GB
Q6_K
2299.2 GB
BF16
5565.6 GB

★ Recommended — best balance of quality and VRAM usage.

Learn more about quantization →

Can I run Kimi K3 on a Mac?

Kimi K3 requires at least 770.2 GB at IQ2_XXS. None of the 146 Macs we list has that much usable memory (the largest has 508.5 GB after macOS overhead).

Can I run Kimi K3 locally?

Yes — Kimi K3 can run locally on consumer hardware. Popular tools include Ollama, LM Studio, and llama.cpp.

What's the download size of Kimi K3?

At Q4_K_M, the download is about 1667.96 GB. The full-precision BF16 version is 5559.86 GB. The smallest option (IQ2_XXS) is 764.48 GB.

Which GPUs can run Kimi K3?

No single consumer GPU has enough VRAM to run Kimi K3 at Q4_K_M (1673.7 GB). Multi-GPU or professional hardware is required.

Which devices can run Kimi K3?

Kimi K3 requires at least 1673.7 GB at Q4_K_M, which exceeds the usable memory of every unified-memory device we list (the largest has 508.5 GB).