All LLM Models
Browse 1475 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Llama 4 Scout 17B 16E Instruct
Meta · 108.6B · runs from 32.9 GB
162.9K
1.4KLlama 4 Scout is the smaller of Meta's first Llama 4 models, a natively multimodal Mixture-of-Experts model with 16 experts, about 109 billion total parameters and 17 billion active per token. This instruction-tuned checkpoint accepts text and images and generates text, and is aimed at assistant-style chat, visual understanding and long-document work. Meta advertises an unusually long context window of up to 10 million tokens, and the model is released under the Llama 4 Community License. Because every expert has to stay in memory, 4-bit quantization still needs roughly 60GB, which means a high-memory workstation, a large unified-memory Mac or a multi-GPU setup rather than a single consumer card.
Gemma 2 2B IT Abliterated
IlyaGusev · 2.6B · runs from 1.6 GB
Gemma 2 2B IT Abliterated is a 2.6B-parameter open language model from IlyaGusev in the Gemma 2 family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 3n E4B IT
Google · 7.8B · runs from 2.4 GB
Gemma 3n E4B IT is the instruction-tuned variant of Google's Gemma 3n E4B, a multimodal model built for on-device use on phones, laptops, and tablets that accepts text, image, audio, and video input and can perform automatic speech recognition and speech translation alongside text chat. It uses Google's MatFormer (Matryoshka Transformer) architecture, which nests a smaller sub-model inside the full network so the same checkpoint can run at reduced effective capacity, and Per-Layer Embedding, which caches embedding parameters to fast local storage instead of holding them all in memory. The "E4B" designation refers to an effective parameter count of around 4 billion at inference, even though the checkpoint's total parameters are larger; either way it is light enough for a single consumer GPU or even a high-end phone. Context length is 32,768 tokens. It is released under Google's Gemma Terms of Use, a custom license permitting broad commercial and research use alongside a prohibited-use policy, and was published in June 2025.
DeepSeek R1
DeepSeek · 684.5B · runs from 192.1 GB
DeepSeek R1 is a groundbreaking reasoning model that uses reinforcement learning to develop chain-of-thought capabilities without relying on supervised fine-tuning. With 684.5 billion total parameters in a mixture-of-experts architecture (only 37 billion active per token), R1 achieves performance competitive with OpenAI's o1 on math, coding, and complex reasoning benchmarks while remaining fully open-weight. Running the full R1 locally is a serious undertaking, requiring well over 300 GB of VRAM at full precision, though quantized versions bring it within reach of multi-GPU setups. For users who want R1-level reasoning on more modest hardware, DeepSeek also released a family of distilled models that pack R1's reasoning patterns into smaller dense architectures.
Qwen3.8 27B OBLITERATED
OBLITERATUS · 27.8B · runs from 12.6 GB
Qwen3.8 27B OBLITERATED is a 27.8B-parameter open language model from OBLITERATUS in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phonellm Alpha 1
pipecat-ai · 31.6B · runs from 9.1 GB
Phonellm Alpha 1 is a 31.6B-parameter open language model from pipecat-ai. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NeoHorse 1 4B
TokenRhythm · 4.2B · runs from 2.3 GB
NeoHorse 1 4B is a 4.2B-parameter open language model from TokenRhythm. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 E2B IT Uncensored
TrevorJS · 5.1B · runs from 2.5 GB
Gemma 4 E2B IT Uncensored is a 5.1B-parameter open language model from TrevorJS in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 VL 32B Instruct
Alibaba · 33.5B · runs from 10.0 GB
Qwen2.5 VL 32B Instruct is a 33.5-billion-parameter vision-language model from Alibaba's Qwen team, able to process images and text together for document parsing, chart reading, and visual question answering. It was tuned with reinforcement learning on top of the original Qwen2.5-VL release for more detailed, better-formatted answers and sharper accuracy on math and visual-logic problems. At this size, local inference needs quantization and a single high-end 24-32GB-class consumer or workstation GPU rather than lower-end hardware. The model supports a 128K token context window, enough for lengthy documents or multi-image inputs. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and was published in March 2025 as a mid-sized addition to the Qwen2.5-VL lineup, between the smaller 7B model and the flagship 72B version.
Qwen2 VL 2B Instruct
Alibaba · 2.2B · runs from 1.1 GB
Qwen2 VL 2B Instruct is a 2.2-billion-parameter vision-language model from Alibaba's Qwen2-VL series, able to process images, multi-image comparisons, and video alongside text prompts. It targets visual question answering, document and chart reading, and basic agentic tasks such as interpreting a screenshot to plan a next action. Its small size makes it well suited to laptops and even some phones, running comfortably on modest consumer hardware once quantized. The model supports a 32K token context window, enough for moderate documents or extended chat. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and was published in August 2024. Qwen2-VL introduced Naive Dynamic Resolution and Multimodal Rotary Position Embedding, letting it handle arbitrary image resolutions and understand videos well over twenty minutes long.
Mixtral 8x7B Instruct v0.1
Mistral AI · 46.7B · runs from 20.4 GB
Mixtral 8x7B Instruct v0.1 is Mistral AI's flagship Mixture-of-Experts model, combining eight expert networks of 7 billion parameters each for a 46.7B total weight count while activating only about 12.9 billion parameters per token. This sparse architecture delivers performance that rivals much larger dense models at a fraction of the inference cost, excelling across reasoning, code generation, and multilingual tasks. Because the full weights must still be loaded into memory, you will need around 24–48 GB of VRAM depending on quantization level, making it best suited for multi-GPU desktop setups or high-VRAM workstation cards. If your hardware can accommodate it, Mixtral offers one of the best performance-per-active-parameter ratios available for local deployment.
Qwen2.5 Coder 0.5B Instruct
Alibaba · 494M · runs from 0.5 GB
Qwen2.5 Coder 0.5B Instruct is a 494M-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 VL 235B A22B Thinking
Alibaba · 235.7B · runs from 91.8 GB
Qwen3-VL-235B-A22B-Thinking is the reasoning-enhanced edition of Alibaba's Qwen3-VL vision-language family, a mixture-of-experts model with roughly 22.8 billion active parameters out of about 235.7 billion total, that reasons step by step before answering on both text and visual inputs. Compared to the Instruct edition, Thinking emphasizes deeper multimodal reasoning for STEM, math, and causal analysis. The model can operate PC and mobile GUIs as a visual agent, generate code from images or video, judge spatial relationships and 3D layouts, and read OCR text in 32 languages. Given its scale, it needs a multi-GPU server to run even quantized. Context length is 262,144 tokens natively, expandable to 1,048,576 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2025.
Qwen3 235B A22B
Alibaba · 235.1B · runs from 100.4 GB
Qwen3 235B A22B is the largest model in Alibaba Cloud's Qwen 3 series, a Mixture of Experts (MoE) model with 235 billion total parameters and approximately 22 billion active parameters per forward pass. The MoE architecture enables it to deliver performance competitive with the best available open-weight models while requiring significantly less compute per token than a comparably sized dense model. It supports hybrid thinking mode for flexible chain-of-thought reasoning. Due to its massive total parameter count, running Qwen3 235B A22B locally requires substantial VRAM to load all expert weights, typically needing multiple high-end professional GPUs even at reduced precision. In heavily quantized formats it becomes accessible on workstation-class multi-GPU setups. Released under the Apache 2.0 license.
Qwen2.5 Math 1.5B Instruct
Alibaba · 1.5B · runs from 1 GB
Qwen2.5 Math 1.5B Instruct is a 1.5B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GLM 4.7 Flash REAP 23B A3B
Cerebras · 23.0B · runs from 7.4 GB
GLM 4.7 Flash REAP 23B A3B is a 23.0B-parameter open language model from Cerebras in the GLM 4 family. It supports a context window of up to 202,752 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NeoHorse 1 9B
TokenRhythm · 9.0B · runs from 4.4 GB
NeoHorse 1 9B is a 9.0B-parameter open language model from TokenRhythm. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mistral Small 3.1 24B Instruct 2503
Mistral AI · 24.0B · runs from 7.3 GB
Mistral Small 3.1 24B Instruct 2503 is a 24-billion-parameter model from Mistral AI, the French AI lab, built on the earlier text-only Mistral Small 3 with added support for image input alongside text. It can reason about images in the same conversation as written prompts, useful for document understanding and multimodal chat. At 24 billion parameters, it needs quantization and a single high-end 24GB-class consumer or workstation GPU for local inference rather than budget hardware. The model supports a 128K token context window for long documents or extended conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in March 2025, it added vision understanding and a longer context window to the earlier text-only Mistral Small while keeping the same 24B parameter budget.
Nemotron 3 Nano Omni 30B A3B Reasoning BF16
NVIDIA · 33.0B · runs from 10.0 GB
Nemotron 3 Nano Omni 30B A3B Reasoning BF16 is a 33.0B-parameter open language model from NVIDIA in the Nemotron family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
SmolVLM2 2.2B Instruct
Hugging Face · 2.2B · runs from 0.7 GB
SmolVLM2-2.2B Instruct is Hugging Face's 2.2-billion-parameter vision-language model, the largest member of the SmolVLM2 family, built to process interleaved text, images, and video within a single conversation. Beyond captioning and visual question answering, it is trained on additional video-instruction data for tasks like summarizing clips or answering questions about video content. At this size it still runs on a single consumer GPU, useful for on-device, resource-constrained deployment. The model supports a context window of 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2025 alongside smaller 500M and 256M SmolVLM2 siblings. Unlike those variants, which favor raw efficiency, the 2.2B model is tuned to be the most capable of the three on both image and video benchmarks.
SmolLM2 360M Instruct
Hugging Face · 362M · runs from 0.5 GB
SmolLM2 360M Instruct is an instruction-tuned model from Hugging Face that occupies the sweet spot between the 135M and 1.7B entries in the SmolLM2 lineup. At 360 million parameters, it offers noticeably better coherence and instruction-following ability than the smallest variants while still running comfortably on virtually any modern GPU or even on CPU. This model is well suited for on-device assistants, embedded applications, and rapid prototyping where you need real conversational ability without dedicating significant hardware resources. It handles short-form generation, summarization, and basic reasoning tasks with reasonable quality.
Kimi Dev 72B
Moonshot AI · 72.7B · runs from 21.0 GB
Kimi Dev 72B is Moonshot AI's developer-focused model built on the Qwen2.5-72B architecture, specifically optimized for coding tasks, tool use, and agentic workflows. It combines strong general-purpose chat abilities with specialized developer capabilities, making it a compelling choice for software engineering assistance. At 72 billion parameters it requires substantial hardware, typically needing 40+ GB of VRAM at 4-bit quantization, which puts it in reach of dual consumer GPU setups or single professional cards like the A100 or RTX 6000 Ada. If you are primarily looking for a local coding assistant with strong reasoning skills, Kimi Dev is a top-tier option in the 70B class.
Mistral Medium 3.5 128B
Mistral AI · 127.7B · runs from 55.3 GB
Mistral Medium 3.5 128B is a 127.7B-parameter open language model from Mistral AI in the Mistral family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
MiniCPM V 4.6
OpenBMB · 1.3B · runs from 0.9 GB
MiniCPM-V 4.6 is OpenBMB's 1.3-billion-parameter vision-language model, pairing a SigLIP2 image encoder with a small Qwen3.5-based backbone to read images, multi-image sequences, and video alongside text. It targets on-device and edge deployment above all else, with adapted builds for iOS, Android, and HarmonyOS, small enough to run on phones and any modern laptop without heavy hardware, quantized or not. The model offers a 262K token context window, unusually long for its size. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2026, introducing visual token compression that cuts visual-encoding computation by more than half versus earlier MiniCPM-V releases.
Cydonia 24B V4.3
TheDrummer · 23.6B · runs from 7.8 GB
Cydonia 24B V4.3 is a 23.6B-parameter open language model from TheDrummer. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 Omni 3B
Alibaba · 5.5B · runs from 1.7 GB
Qwen2.5-Omni-3B is Alibaba's 5.5-billion-parameter end-to-end omni-modal model, built to perceive text, images, audio, and video and respond with both text and natural streaming speech. It uses a Thinker-Talker architecture with a time-aligned position embedding (TMRoPE) that synchronizes video frames with audio, enabling real-time voice and video chat with chunked input and immediate output. As the smaller of the two Omni checkpoints, it runs on a single consumer GPU once quantized. Its language backbone supports a 32,768 token context window. It is released under Alibaba's Qwen Research License, which permits only non-commercial research and evaluation, unlike the Apache 2.0 used for Qwen's text-only models. Published in April 2025, about a month after the larger Qwen2.5-Omni-7B, it brings real-time multimodal chat to smaller-scale deployments.
Granite 4.0 H Tiny
IBM · 6.9B · runs from 2.4 GB
Granite 4.0 H Tiny is a 6.9B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
OlmOCR 2 7B 1025
Allen AI · 8.3B · runs from 2.7 GB
olmOCR-2-7B-1025 is Allen AI's 8.3-billion-parameter vision-language model, fine-tuned from Qwen2.5-VL-7B-Instruct specifically for document OCR rather than general chat. It converts scanned pages and PDFs into clean text, using GRPO reinforcement learning atop supervised fine-tuning to sharpen accuracy on math, tables, and tricky OCR cases. This is the full-precision BF16 release; Allen AI recommends its FP8 sibling for production use. At this size, it runs on a single mainstream-to-high-end consumer GPU once quantized. It supports a 128,000 token context window inherited from its Qwen2.5-VL base, useful for multi-page documents. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2025, it pairs with Allen AI's olmOCR toolkit, which handles page rendering and retries at scale via vLLM.
Deepseek Coder 6.7B Base
DeepSeek · 6.7B · runs from 3.2 GB
Deepseek Coder 6.7B Base is a 6.7B-parameter open language model from DeepSeek in the DeepSeek Coder family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
MiniCPM5 2B
OpenBMB · 2.5B · runs from 1.2 GB
MiniCPM5-2B is the second model in OpenBMB's MiniCPM5 series, a dense 2.5-billion-parameter causal language model built on a Llama-style architecture. This release is the final version, post-trained with reinforcement learning and on-policy distillation, and is aimed at local assistants, coding agents, tool-use workflows, and reasoning tasks where a small footprint matters, including on-device and edge deployment. It supports a 131,072-token context window, unusually long for its size, and ships under the Apache 2.0 license, allowing free commercial and non-commercial use. At roughly 2.5 billion parameters, it needs well under 2GB of memory at 4-bit quantization, so it runs comfortably on almost any modern GPU or laptop.