Olmo 3 32B Think — Hardware Requirements & GPU Compatibility
Chat
Olmo 3 32B Think is Allen Institute for AI's reasoning-focused model in the fully open Olmo 3 family, trained to produce long chains of thought before answering so it can work through math and coding problems step by step. It is pretrained on Ai2's Dolma 3 corpus and then carried through a full post-training pipeline of supervised fine-tuning, direct preference optimization, and reinforcement learning with verifiable rewards on the Dolci-Think-RL dataset; Ai2 releases the code, intermediate checkpoints, and training logs for each stage, not just the final weights. It sits alongside a 7B Think and a 7B Instruct sibling in the same release. At 32 billion parameters it needs a high-end consumer GPU or a multi-GPU setup once quantized. Context length is 65,536 tokens. It is released under the Apache 2.0 license, though Ai2 asks that it be used for research and educational purposes in line with its Responsible Use Guidelines. It was published in November 2025 and has since been superseded by Olmo 3.1 32B Think.
10.7K downloads
177 likesSpecifications
- Publisher
- Allen AI
- Family
- OLMo
- Parameters
- 32.2B
- Architecture
- Olmo3ForCausalLM
- Context Length
- 65,536 tokens
- Vocabulary Size
- 100,278
- Release Date
- 2025-11-19
- License
- Apache 2.0
Get Started
HuggingFace
How Much VRAM Does Olmo 3 32B Think Need?
Select a quantization to see compatible GPUs below.
| Quantization | Bits | VRAM | + Context | File Size | Quality |
|---|---|---|---|---|---|
| Q2_K | 3.40 | 14.5 GB | 31.2 GB | 13.70 GB | 2-bit quantization with K-quant improvements |
| Q3_K_S | 3.50 | 14.9 GB | 31.6 GB | 14.10 GB | 3-bit small quantization |
| Q3_K_M | 3.90 | 16.6 GB | 33.2 GB | 15.71 GB | 3-bit medium quantization |
| Q4_0 | 4.00 | 16.9 GB | 33.6 GB | 16.12 GB | 4-bit legacy quantization |
| Q4_K_M | 4.80 | 20.2 GB | 36.8 GB | 19.34 GB | 4-bit medium quantization — most popular sweet spot |
| Q5_K_M | 5.70 | 23.8 GB | 40.5 GB | 22.97 GB | 5-bit medium quantization — good quality/size tradeoff |
| Q6_K | 6.60 | 27.4 GB | 44.1 GB | 26.59 GB | 6-bit quantization, very good quality |
| Q8_0 | 8.00 | 33.1 GB | 49.7 GB | 32.23 GB | 8-bit quantization, near-lossless |
Which GPUs Can Run Olmo 3 32B Think?
Q4_K_M · 20.2 GBOlmo 3 32B Think (Q4_K_M) requires 20.2 GB of VRAM to load the model weights. For comfortable inference with headroom for KV cache and system overhead, 27+ GB is recommended. Using the full 66K context window can add up to 16.6 GB, bringing total usage to 36.8 GB. 7 GPUs can run it, including NVIDIA GeForce RTX 5090, NVIDIA GeForce RTX 3090 Ti.
Runs great
— Plenty of headroom
Which Devices Can Run Olmo 3 32B Think?
Q4_K_M · 20.2 GB41 devices with unified memory can run Olmo 3 32B Think, including NVIDIA DGX H100, NVIDIA DGX A100 640GB, Mac Mini M4 Pro (24 GB).
Runs great
— Plenty of headroomDecent
— Enough memory, may be tightWhere to Download Olmo 3 32B Think
Community quantizations of this model — GGUF for llama.cpp, Ollama, and LM Studio, plus AWQ/MLX variants where available.
Benchmarks
Benchmark details →Related Models
Frequently Asked Questions
- How much VRAM does Olmo 3 32B Think need?
Olmo 3 32B Think requires 20.2 GB of VRAM at Q4_K_M, or 65.3 GB at BF16. Full 66K context adds up to 16.6 GB (36.8 GB total).
VRAM = Weights + KV Cache + Overhead
Weights = 32.2B × 4.8 bits ÷ 8 = 19.3 GB
KV Cache + Overhead ≈ 0.9 GB (at 2K context + ~0.3 GB framework)
KV Cache + Overhead ≈ 17.5 GB (at full 66K context)
VRAM usage by quantization
Q4_K_M20.2 GBQ4_K_M + full context36.8 GB- Can NVIDIA GeForce RTX 4090 run Olmo 3 32B Think?
Yes, at Q5_K_M (23.8 GB) or lower. Higher quantizations like Q6_K (27.4 GB) exceed the NVIDIA GeForce RTX 4090's 24 GB.
- What's the best quantization for Olmo 3 32B Think?
For Olmo 3 32B Think, Q4_K_M (20.2 GB) offers the best balance of quality and VRAM usage. Q5_K_S (23 GB) provides better quality if you have the VRAM. The smallest option is IQ2_XXS at 9.7 GB.
VRAM requirement by quantization
IQ2_XXS9.7 GBQ3_K_S14.9 GBQ4_119.0 GBQ4_K_M ★20.2 GBQ5_K_S23.0 GBBF1665.3 GB★ Recommended — best balance of quality and VRAM usage.
- Can I run Olmo 3 32B Think on a Mac?
Olmo 3 32B Think requires at least 9.7 GB at IQ2_XXS, which exceeds the unified memory of most consumer Macs. You would need a Mac Studio or Mac Pro with a high-memory configuration.
- Can I run Olmo 3 32B Think locally?
Yes — Olmo 3 32B Think can run locally on consumer hardware. At Q4_K_M quantization it needs 20.2 GB of VRAM. Popular tools include Ollama, LM Studio, and llama.cpp.
- How fast is Olmo 3 32B Think?
At Q4_K_M, Olmo 3 32B Think can reach ~238 tok/s on AMD Instinct MI350X. On NVIDIA GeForce RTX 4090: ~33 tok/s. Speed depends mainly on GPU memory bandwidth. Real-world results typically within ±20%.
tok/s = (bandwidth GB/s ÷ model GB) × efficiency
Example: NVIDIA B200 → 8000 ÷ 20.2 × 0.65 = ~258 tok/s
Estimated speed at Q4_K_M (20.2 GB)
~258 tok/s~33 tok/s~258 tok/s~238 tok/sReal-world results typically within ±20%. Speed depends on batch size, quantization kernel, and software stack.
- What's the download size of Olmo 3 32B Think?
At Q4_K_M, the download is about 19.34 GB. The full-precision BF16 version is 64.47 GB. The smallest option (IQ2_XXS) is 8.86 GB.
- Which GPUs can run Olmo 3 32B Think?
7 consumer GPUs can run Olmo 3 32B Think at Q4_K_M (20.2 GB). Top options include NVIDIA GeForce RTX 5090, AMD Radeon RX 7900 XTX, NVIDIA GeForce RTX 3090. 1 GPU have plenty of headroom for comfortable inference.
- Which devices can run Olmo 3 32B Think?
41 devices with unified memory can run Olmo 3 32B Think at Q4_K_M (20.2 GB), including AMD Ryzen AI 9 HX 370 (Strix Point) Laptop, ASUS Ascent GX10, Asus ROG Flow Z13 (2025, Ryzen AI Max+ 395, 128 GB), Beelink GTR9 Pro (Ryzen AI Max+ 395, 128 GB). Apple Silicon Macs use unified memory shared between CPU and GPU, making them well-suited for local LLM inference.