All LLM Models
Browse 1214 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Trinity Mini
Arcee AI · 26.1B · runs from 11.5 GB
25.3K
205Trinity Mini is a 26.1B-parameter open language model from Arcee AI. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
DeepSeek R1 Distill Qwen 7B
DeepSeek · 7.6B · runs from 3.0 GB
DeepSeek R1 Distill Qwen 7B compresses the reasoning techniques from DeepSeek's full R1 model into a compact 7.6 billion parameter dense model built on the Qwen 2.5 architecture. Despite its small footprint, it demonstrates surprisingly capable step-by-step reasoning on math and logic problems that would stump many models several times its size. This is one of the most accessible reasoning models available for local use, fitting comfortably on GPUs with 6 GB or more of VRAM when quantized. It strikes a practical balance between genuine chain-of-thought reasoning ability and the hardware constraints of a typical consumer setup.
Qwen3.6 27B Uncensored Heretic v2 Native MTP Preserved
llmfan46 · 27.4B · runs from 12.4 GB
Qwen3.6 27B Uncensored Heretic v2 Native MTP Preserved is a 27.4B-parameter open language model from llmfan46 in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 3.2 3B
Meta · 3.2B · runs from 1.5 GB
Meta Llama 3.2 3B is a 3.2-billion parameter base (pretrained) model from Meta's Llama 3.2 family. It supports a 128K token context window and is intended for fine-tuning, research, and custom applications rather than direct conversational use. The model provides a good balance between capability and efficiency at the small model scale. It is popular as a foundation for community fine-tunes and domain-specific adaptations. Released under the Llama 3.2 Community License.
Qwen3 VL 32B Instruct
Alibaba · 33.4B · runs from 9.8 GB
Qwen3 VL 32B Instruct is Alibaba's largest dense model in the initial Qwen3-VL lineup, a 33.4-billion-parameter vision-language model built to process images and text in one pass. It handles image description, visual question answering, and document understanding, and its visual-agent tuning lets it read GUI screenshots and reason about on-screen elements. Local inference calls for quantization and a capable GPU, fitting on a single high-end consumer or workstation card. The model supports a 262,144 token context window, enough for long documents or extended multi-turn conversations. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2025 alongside the 2B and 8B models, it shares the family's long native context and video-understanding capabilities, giving stronger multimodal reasoning than the smaller variants.
Gemma 4 26B A4B IT Uncensored
TrevorJS · 25.8B · runs from 11.6 GB
Gemma 4 26B A4B IT Uncensored is a 25.8B-parameter open language model from TrevorJS in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Ornith 1.5 35B A3B
ornith-ai · 36.0B · runs from 10.3 GB
Ornith-1.5-35B-A3B is the mid-size model in Ornith AI's Ornith-1.5 family, a 36-billion-parameter Mixture-of-Experts model that activates only about 3 billion parameters per token. Like its siblings, it is built for agentic coding and inherits the family's self-improving training loop, which uses reinforcement learning to jointly refine training tasks, agent scaffolding, and solution rollouts rather than relying on a fixed, hand-built curriculum. It supports a 262,144-token context window and is released under the MIT license. Because MoE models keep all experts in memory, 4-bit quantization needs roughly 21GB, fitting a single GPU in the 16-24GB range.
Mistral Small 3.2 24B Instruct 2506
Mistral AI · 24.0B · runs from 7.3 GB
Mistral-Small-3.2-24B-Instruct-2506 is a dense 24-billion-parameter vision-language model from Mistral AI, a minor refinement of Mistral-Small-3.1-24B-Instruct-2503. The update focuses on following precise instructions more reliably, cutting down on repetitive or runaway generations, and making function calling more robust, while still handling both text and image inputs for tasks like chart and document understanding. It supports a 131,072-token context window and is released under the Apache 2.0 license. At this size, 4-bit quantization needs roughly 14GB of memory, so it runs on a single consumer GPU in the 16-24GB range, such as an RTX 4090.
LFM2.5 350M
Liquid AI · 354M · runs from 0.5 GB
LFM2.5 350M is a 354M-parameter open language model from Liquid AI in the LFM2.5 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 4B DFlash B16
z-lab · 537M · runs from 0.6 GB
Qwen3 4B DFlash B16 is a 537M-parameter open language model from z-lab in the Qwen 3 family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 4 Mini Reasoning
Microsoft · 3.8B · runs from 1.6 GB
Phi-4-mini-reasoning is Microsoft's compact 3.8-billion-parameter model in the Phi-4 family, fine-tuned specifically for multi-step, logic-intensive mathematical reasoning using synthetic math data distilled from a larger teacher model. It targets tasks like formal proof generation, symbolic computation, and advanced word problems, and is designed to run in memory- and latency-constrained environments such as educational tools or embedded tutoring systems, rather than for general knowledge or open-domain chat. It supports a 128K token context window and is released under the MIT license. At 3.8 billion parameters, it runs comfortably on almost any modern laptop or consumer GPU, even at higher precision, making it one of the more accessible reasoning-focused models to run locally.
TinyLlama 1.1B Chat v1.0
TinyLlama · 1.1B · runs from 0.8 GB
TinyLlama 1.1B Chat is a 1.1-billion parameter chat model built on the Llama 2 architecture and trained on approximately 3 trillion tokens, an unusually large dataset for a model of its size. The TinyLlama project demonstrated that small models can achieve strong performance when given sufficient training compute, making it a standout in the sub-2B parameter class. The Chat variant is fine-tuned for conversational use and runs on virtually any modern GPU, including entry-level cards with 4GB of VRAM or less. It is a practical choice for lightweight local inference, edge deployment, and experimentation where hardware resources are limited.
Qwen3.5 9B DFlash
z-lab · 1.3B · runs from 0.9 GB
Qwen3.5 9B DFlash is a 1.3B-parameter open language model from z-lab in the Qwen 3.5 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.5 4B DFlash
z-lab · 634M · runs from 0.6 GB
Qwen3.5 4B DFlash is a 634M-parameter open language model from z-lab in the Qwen 3.5 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 31B IT Uncensored Heretic
llmfan46 · 31.3B · runs from 14.9 GB
Gemma 4 31B IT Uncensored Heretic is a 31.3B-parameter open language model from llmfan46 in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Ornith 1.5 9B
ornith-ai · 9.7B · runs from 3.2 GB
Ornith-1.5-9B is the lightest model in Ornith AI's Ornith-1.5 family, a dense 9.7-billion-parameter model built for agentic coding tasks and designed for efficient single-GPU deployment. It shares the family's self-improving training approach, in which reinforcement learning jointly optimizes the tasks, agent scaffolding, and solution rollouts used to train it, and a quantized variant is available for edge and mobile deployment. It offers a 262,144-token context window and is released under the MIT license. At roughly 9.7 billion parameters, 4-bit quantization needs only about 5.6GB of memory, so it runs comfortably on a single mid-range consumer GPU.
Diffusiongemma 26B A4B IT
Google · 25.8B · runs from 11.6 GB
Diffusiongemma 26B A4B IT takes a different approach from the rest of the Gemma 4 family: rather than generating tokens autoregressively, it uses a block-diffusion architecture. It still follows a mixture-of-experts design, with roughly 26 billion total parameters and about 4 billion active per token. The active-parameter figure is the more relevant one for inference cost, while the total parameter count determines memory needs, putting local use in single high-end consumer GPU territory once quantized. It also accepts image input alongside text for multimodal use. The model offers a 256K token context window and is released under the Apache 2.0 license. Published on June 9, 2026, it represents Google's exploration of diffusion-based language modeling within the Gemma 4 generation, alongside the more conventional autoregressive Gemma 4 variants also released that year.
Bonsai 1.7B Unpacked
prism-ml · 1.7B · runs from 1.3 GB
Bonsai 1.7B Unpacked is a 1.7B-parameter open language model from prism-ml. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
DeepSeek R1 Distill Qwen 32B
DeepSeek · 32.8B · runs from 9.8 GB
DeepSeek R1 Distill Qwen 32B takes the reasoning capabilities developed in the full 684.5B R1 model and distills them into the 32.8 billion parameter Qwen 2.5 architecture. The result is a dense model that punches well above its weight class on math, science, and coding reasoning tasks, often matching models two to three times its size. At around 32.8 billion parameters, this model fits comfortably on a single high-end consumer GPU when quantized to 4-bit precision, making it one of the most capable reasoning models you can run on a desktop workstation.
Ministral 3 14B Reasoning 2512
Mistral AI · 13.9B · runs from 6.7 GB
Ministral 3 14B Reasoning 2512 is a 13.9B-parameter open language model from Mistral AI in the Mistral family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron Nano 12B v2
NVIDIA · 12.3B · runs from 5.8 GB
NVIDIA Nemotron Nano 12B v2 is a 12.3B-parameter open language model from NVIDIA in the Nemotron family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 VL 2B Instruct
Alibaba · 2.1B · runs from 1.1 GB
Qwen3 VL 2B Instruct is Alibaba's smallest model in the Qwen3-VL lineup, a 2.1-billion-parameter vision-language model built to handle images and text together. It performs image captioning, visual question answering, and document reading, and its visual-agent tuning lets it interpret GUI screenshots for simple automation. Its small size suits on-device and edge deployment, running comfortably on modest consumer GPUs or even some laptops once quantized. The model supports a 262,144 token context window, enough for lengthy documents or multi-turn conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in October 2025 alongside the 8B and 32B models, it targets edge and mobile use cases, trading some visual reasoning depth for a much smaller footprint.
Llama 3.3 70B Instruct Abliterated
huihui-ai · 70.6B · runs from 20.4 GB
Llama 3.3 70B Instruct Abliterated is a 70.6B-parameter open language model from huihui-ai in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 30B A3B Thinking 2507
Alibaba · 30.5B · runs from 8.8 GB
Qwen3 30B A3B Thinking 2507 is the reasoning-focused variant of Alibaba's 30-billion-parameter mixture-of-experts model, updated in July 2025. Like its instruct sibling, it activates only about 3 billion parameters per token, keeping resource demands low while enabling multi-step reasoning and chain-of-thought problem solving. This thinking variant is designed for tasks that benefit from deliberate, step-by-step logic such as math, coding puzzles, and analytical questions. Its efficient MoE design means users with modest GPUs can still access strong reasoning capabilities without needing datacenter-class hardware.
Qwen3.6 35B A3B Uncensored Heretic Native MTP Preserved
llmfan46 · 35.1B · runs from 15.3 GB
Qwen3.6 35B A3B Uncensored Heretic Native MTP Preserved is a 35.1B-parameter open language model from llmfan46 in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 Omni 30B A3B Instruct
Alibaba · 35.3B · runs from 16.5 GB
Qwen3-Omni-30B-A3B-Instruct is Alibaba's flagship omni-modal model, a mixture-of-experts Thinker-Talker system that processes text, images, audio, and video and responds with real-time streaming text and speech. Its text backbone routes across 128 experts with 8 active per token, roughly 3 billion active parameters out of around 35 billion total, since it keeps both the "thinker" and the speech-generating "talker" plus its audio codec resident in memory. It supports 119 text languages, 19 spoken input languages, and 10 spoken output languages, and needs a high-end consumer GPU or multi-GPU setup once quantized. Its language backbone supports a 65,536 token context window. It is released under the Apache 2.0 license, and was published in September 2025 as the instruct counterpart to Qwen3-Omni-30B-A3B-Thinking, which drops the talker to focus on text output with chain-of-thought reasoning.
MN 12B Mag Mell R1
inflatebot · 12.2B · runs from 4.1 GB
MN 12B Mag Mell R1 is a 12.2B-parameter open language model from inflatebot. It supports a context window of up to 1,024,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Deepseek Coder 6.7B Instruct
DeepSeek · 6.7B · runs from 4.2 GB
DeepSeek Coder 6.7B Instruct is a first-generation code-specialized model trained on a large corpus of source code and programming-related data. At 6.7 billion parameters, it provides solid code completion, generation, and explanation capabilities across popular programming languages while remaining small enough to run on most consumer GPUs. While newer models in the DeepSeek lineup have surpassed it in raw capability, this model remains a practical choice for users who need a lightweight local coding assistant with minimal hardware requirements. It runs well on GPUs with as little as 6 GB of VRAM when quantized.
Llama 2 7B Chat HF
Meta · 6.7B · runs from 3.1 GB
Meta Llama 2 7B Chat is a 7-billion parameter instruction-tuned model from Meta's Llama 2 family, optimized for dialogue use cases. It was fine-tuned using supervised fine-tuning and RLHF on top of the Llama 2 7B base model, with a 4K token context window. This model is suitable for basic conversational AI tasks and runs efficiently on consumer GPUs. While newer Llama generations offer improved performance, Llama 2 7B Chat remains a well-understood and widely-supported option for local inference. Released under the Llama 2 Community License.
Hy MT2 7B
Tencent · 8.0B · runs from 2.8 GB
Hy-MT2-7B is Tencent's mid-size entry in the Hy-MT2 family of dedicated, "fast-thinking" multilingual translation models, alongside 1.8B and 30B-A3B (mixture-of-experts) siblings. Rather than being a general-purpose chat assistant, it is trained specifically for translation across 33 languages and to follow multilingual translation instructions, and the card reports it outperforming open models such as DeepSeek-V4-Pro and Kimi K2.6 in fast-thinking mode. Tencent also ships FP8 and GGUF quantized builds for on-device use with llama.cpp. At roughly 8 billion dense parameters, it fits on a single consumer GPU once quantized. Context length is 262,144 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2026, alongside the open-sourced IFMTBench instruction-following translation benchmark.
