Cogito 671B V2.1 — Hardware Requirements & GPU Compatibility
Chat
Cogito v2.1 671B is Deep Cogito's instruction-tuned mixture-of-experts model, with 671 billion total and 37 billion active parameters, fine-tuned from DeepSeek-V3-Base. It is a hybrid reasoning model that can answer directly or reflect before answering, and was trained with Iterated Distillation and Amplification. The card says it is optimized for coding, STEM, instruction following and tool calling, and trained in over 30 languages. The BF16 checkpoint takes about 1.3 TB, and the card says it needs at least 8 B200 GPUs or 16 H200s, with an FP8 build for 8 H200s, so it is a datacenter-class model rather than a local one. The card states a 128k context length, while the configuration lists 163,840 tokens. It is released under the MIT license, permitting unrestricted commercial and research use. Published in October 2025, it is the 671B member of the Cogito v2.1 family.
523 downloads
34 likesSpecifications
- Publisher
- deepcogito
- Parameters
- 671.0B
- Architecture
- DeepseekV3ForCausalLM
- Context Length
- 163,840 tokens
- Vocabulary Size
- 128,815
- Release Date
- 2025-10-24
- License
- MIT
Get Started
HuggingFace
How Much VRAM Does Cogito 671B V2.1 Need?
Select a quantization to see compatible GPUs below.
| Quantization | Bits | VRAM | + Context | File Size | Quality |
|---|---|---|---|---|---|
| Q2_K | 3.40 | 289.1 GB | 572.0 GB | 285.18 GB | 2-bit quantization with K-quant improvements |
| Q3_K_S | 3.50 | 297.4 GB | 580.4 GB | 293.56 GB | 3-bit small quantization |
| Q3_K_M | 3.90 | 331.0 GB | 614.0 GB | 327.11 GB | 3-bit medium quantization |
| Q4_0 | 4.00 | 339.4 GB | 622.4 GB | 335.50 GB | 4-bit legacy quantization |
| Q4_K_M | 4.80 | 406.5 GB | 689.5 GB | 402.60 GB | 4-bit medium quantization — most popular sweet spot |
| Q5_K_M | 5.70 | 482.0 GB | 764.9 GB | 478.09 GB | 5-bit medium quantization — good quality/size tradeoff |
| Q6_K | 6.60 | 557.5 GB | 840.4 GB | 553.58 GB | 6-bit quantization, very good quality |
| Q8_0 | 8.00 | 674.9 GB | 957.9 GB | 671.00 GB | 8-bit quantization, near-lossless |
Which GPUs Can Run Cogito 671B V2.1?
Q4_K_M · 406.5 GBCogito 671B V2.1 (Q4_K_M) requires 406.5 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 529+ GB is recommended. Using the full 164K context window can add up to 283.0 GB, bringing total usage to 689.5 GB. No single GPU has enough memory — multi-GPU or cluster setups are needed.
Which Devices Can Run Cogito 671B V2.1?
Q4_K_M · 406.5 GB2 devices with unified memory can run Cogito 671B V2.1, including NVIDIA DGX H100, NVIDIA DGX A100 640GB.
Runs great
— Plenty of headroomWhere to Download Cogito 671B V2.1
Community quantizations of this model — GGUF for llama.cpp, Ollama, and LM Studio, plus AWQ/MLX variants where available.
Benchmarks
Benchmark details →Frequently Asked Questions
- How much VRAM does Cogito 671B V2.1 need?
Cogito 671B V2.1 requires 406.5 GB of VRAM at Q4_K_M, or 1345.9 GB at BF16. Full 164K context adds up to 283.0 GB (689.5 GB total).
VRAM = Weights + KV Cache + Overhead
Weights = 671.0B × 4.8 bits ÷ 8 = 402.6 GB
KV Cache + Overhead ≈ 3.9 GB (at 2K context + ~0.3 GB framework)
KV Cache + Overhead ≈ 286.9 GB (at full 164K context)
VRAM usage by quantization
Q4_K_M406.5 GBQ4_K_M + full context689.5 GB- Can NVIDIA GeForce RTX 5090 run Cogito 671B V2.1?
No — Cogito 671B V2.1 requires at least 188.4 GB at IQ2_XXS, which exceeds the NVIDIA GeForce RTX 5090's 32 GB of VRAM.
- What's the best quantization for Cogito 671B V2.1?
For Cogito 671B V2.1, Q4_K_M (406.5 GB) offers the best balance of quality and VRAM usage. Q5_K_S (465.2 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 188.4 GB.
VRAM requirement by quantization
IQ2_XXS188.4 GBQ3_K_S297.4 GBIQ4_NL381.3 GBQ4_K_M ★406.5 GBQ5_K_S465.2 GBBF161345.9 GB★ Recommended — best balance of quality and VRAM usage.
- Can I run Cogito 671B V2.1 on a Mac?
Cogito 671B V2.1 requires at least 188.4 GB at IQ2_XXS, which exceeds the unified memory of most consumer Macs. You would need a Mac Studio or Mac Pro with a high-memory configuration.
- Can I run Cogito 671B V2.1 locally?
Yes — Cogito 671B V2.1 can run locally on consumer hardware. At Q4_K_M quantization it needs 406.5 GB of VRAM. Popular tools include Ollama, LM Studio, and llama.cpp.
- What's the download size of Cogito 671B V2.1?
At Q4_K_M, the download is about 402.60 GB. The full-precision BF16 version is 1342.00 GB. The smallest option (IQ2_XXS) is 184.53 GB.
- Which GPUs can run Cogito 671B V2.1?
No single consumer GPU has enough VRAM to run Cogito 671B V2.1 at Q4_K_M (406.5 GB). Multi-GPU or professional hardware is required.
- Which devices can run Cogito 671B V2.1?
3 devices with unified memory can run Cogito 671B V2.1 at Q4_K_M (406.5 GB), including Mac Studio (M3 Ultra, 512GB), NVIDIA DGX A100 640GB, NVIDIA DGX H100. Apple Silicon Macs use unified memory shared between CPU and GPU, making them well-suited for local LLM inference.