All LLM Models
Browse 1475 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Hy3
Tencent · 298.8B · runs from 127.6 GB
12.3K
971Hy3 is Tencent's flagship Hunyuan 3 model, a mixture-of-experts design with roughly 299 billion total parameters and about 21 billion active per token. It combines fast and slow "thinking" modes in one architecture for reasoning, coding, and agent workloads. Only the active parameters compute per token, cutting cost versus a same-sized dense model, but all 299 billion weights must sit in memory, putting it in server-class territory; most people use a hosted endpoint, and local use needs multiple GPUs. The model supports a 262K token context window. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in July 2026, Hy3 is integrated across Tencent's products, including its CodeBuddy assistant.
Mistral 7B Instruct v0.3
Mistral AI · 7.2B · runs from 2.7 GB
Mistral 7B Instruct v0.3 is the latest instruction-tuned release of Mistral AI's original 7-billion-parameter model, delivering meaningful improvements in instruction following, function calling, and multilingual support over its predecessors. With an extended 32K-token vocabulary and refined chat capabilities, v0.3 remains one of the most capable sub-10B models available. At 7.2 billion parameters it sits comfortably in the sweet spot for local inference, running well on GPUs with 6–8 GB of VRAM at full precision and even on 4 GB cards with 4-bit quantization. It is an excellent default choice for anyone getting started with local LLMs who wants strong conversational performance without heavy hardware.
Qwen2.5 14B
Alibaba · 14.8B · runs from 6.8 GB
Qwen2.5 14B is a 14.8B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Jan v3 4B Base Instruct
janhq · 4.4B · runs from 2.4 GB
Jan v3 4B Base Instruct is a 4.4B-parameter open language model from janhq. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GPT OSS 120B
OpenAI · 116.8B · runs from 50.1 GB
GPT-OSS 120B is the larger of OpenAI's open-source model releases, bringing 120.4 billion parameters of GPT-lineage capability to the open-weight ecosystem. It represents near-frontier performance across reasoning, knowledge, code generation, and conversational tasks, rivaling top proprietary offerings in many benchmarks. Running this model locally is a serious hardware commitment, typically requiring multiple high-VRAM GPUs or a professional-grade setup with 80+ GB of combined VRAM even at aggressive quantization levels. It is best suited for enthusiasts with multi-GPU rigs or workstation hardware who want the strongest possible local model from OpenAI's catalog.
Gemma 4 31B IT Scotoma 2
ReadyArt · 31.3B · runs from 13.7 GB
Gemma 4 31B IT Scotoma 2 is a 31.3B-parameter open language model from ReadyArt in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 0.5B Instruct
Alibaba · 494M · runs from 0.5 GB
Qwen2.5 0.5B Instruct is the smallest instruction-tuned model in Alibaba Cloud's Qwen 2.5 family, with just 494 million parameters. It is designed for ultra-lightweight deployment scenarios where minimal hardware resources are available, running comfortably on virtually any modern GPU or even CPU-only configurations. Despite its tiny footprint, the model supports a 128K token context window and can handle basic chat, simple summarization, and lightweight instruction following. It is primarily useful for edge deployment, experimentation, and prototyping where model size is a critical constraint. Released under the Apache 2.0 license.
Qwen3 Next 80B A3B Instruct
Alibaba · 81.3B · runs from 22.8 GB
Qwen3 Next 80B A3B Instruct is a Mixture of Experts (MoE) model from Alibaba Cloud's Qwen 3 series, with approximately 81.3 billion total parameters and around 3 billion active parameters per forward pass. This extreme ratio between total and active parameters allows the model to encode extensive knowledge across its expert layers while maintaining very fast per-token inference, making it an unusually efficient design for its capability level. The model is instruction-tuned for general-purpose chat and requires VRAM proportional to its full 80B parameter count for weight loading, typically needing high-VRAM GPUs or quantized multi-GPU setups. Its low active parameter count results in fast generation speeds despite the large total model size. Released under the Apache 2.0 license.
Gemma 2 2B IT
Google · 2.6B · runs from 0.9 GB
Google Gemma 2 2B IT is a 2-billion parameter instruction-tuned model from Google's Gemma 2 family, the smallest variant in the Gemma 2 series. It is designed for efficient local inference on resource-constrained hardware, handling basic conversational tasks and simple instruction following at minimal compute cost. The model can run on GPUs with as little as 4GB of VRAM when quantized, and even on CPU-only setups. Released under the Gemma license.
Agents A1 4B
InternScience · 4.5B · runs from 2 GB
Agents A1 4B is a 4.5B-parameter open language model from InternScience in the Agents-A1 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mellum2 12B A2.5B Instruct
JetBrains · 12.1B · runs from 4.0 GB
Mellum2 12B A2.5B Instruct is JetBrains' 12-billion-parameter mixture-of-experts coding model, with about 2.4 billion parameters active per token across 64 experts. Because only a fraction of the weights are computed for each token, it runs noticeably faster than a dense model of similar size, while all the experts still need to fit in memory. Unlike JetBrains' earlier Mellum, a narrow fill-in-the-middle completion model, Mellum2 is a full coding assistant that can generate and edit code, call tools, hold multi-turn conversations, and reason through problems. Its modest active-parameter count makes it practical to run on a single mainstream consumer GPU once quantized. The model supports a 131K token context window and is released under the Apache 2.0 license, allowing unrestricted commercial and research use. It was published in May 2026.
Ling 3.0 Tiny
Inclusion AI · 7.9B · runs from 2.8 GB
Ling 3.0 Tiny is InclusionAI's smallest mixture-of-experts model in Ant Group's Ling 3.0 family, totaling roughly 7.9 billion parameters with about 1.4 billion active per token. It shares the family's hybrid-linear-attention MoE architecture, also used at far larger scale in Ling-3.0-Flash, scaled down here for lightweight chat and agent use. Only the active parameters compute per token, so it responds quickly while still needing its full parameter set in memory; under 8 billion total, it runs on a mainstream consumer GPU once quantized. The model provides a 131K token context window for extended conversations or agent traces. It is released under the MIT license, allowing unrestricted commercial and research use. Published in August 2026, it extends Ant Group's Ling 3.0 lineup, built for efficient agent workloads.
Yi Coder 9B Chat
01.AI · 8.8B · runs from 3.1 GB
Yi Coder 9B Chat is a 8.8B-parameter open language model from 01.AI in the Yi family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 3.5 Mini Instruct
Microsoft · 3.8B · runs from 2.3 GB
Phi 3.5 Mini Instruct is a 3.8B-parameter open language model from Microsoft in the Phi 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.6 35B A3B Claude 4.6 Opus Reasoning Distilled
hesamation · 36.0B · runs from 15.7 GB
Qwen3.6 35B A3B Claude 4.6 Opus Reasoning Distilled is a 36.0B-parameter open language model from hesamation in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Meta Llama 3 8B Instruct
Meta · 8.0B · runs from 2.6 GB
Meta Llama 3 8B Instruct is the instruction-tuned version of Meta's Llama 3 8B base model, with 8 billion parameters. It is fine-tuned for dialogue and chat use cases using supervised fine-tuning and RLHF, making it ready for conversational applications out of the box. The model supports an 8K token context window and performs well across coding, reasoning, and general knowledge tasks. Its efficient size makes it one of the most popular models for local inference on consumer hardware. Released under the Meta Llama 3 Community License.
Qwen3.5 397B A17B
Alibaba · 403.4B · runs from 807.2 GB
Qwen3.5 397B A17B is a 403.4B-parameter open language model from Alibaba in the Qwen 3.5 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 Next 80B A3B Thinking
Alibaba · 81.3B · runs from 22.8 GB
Qwen3 Next 80B A3B Thinking is Alibaba's 81-billion-parameter mixture-of-experts model, with about 3 billion parameters active per token (the A3B in its name), drawn from 512 experts with only 10 routed at a time. Because only the active experts run per token, inference is far cheaper than a dense model this size, though the full weight set still needs to fit in memory. It combines mostly-linear Gated DeltaNet attention with periodic full attention layers, and this Thinking variant always reasons through a visible chain of thought before answering. Running it locally needs multiple GPUs or a large unified-memory machine. It supports a 262K token context window, is released under the Apache 2.0 license, and was published in September 2025 as part of the Qwen3-Next line introducing this hybrid-attention architecture.
Medgemma 27B Text IT
Google · 27.0B · runs from 59.4 GB
Google MedGemma 27B Text IT is a 27-billion parameter instruction-tuned model specialized for the medical domain, built on the Gemma architecture by Google. It is fine-tuned on medical and clinical text data to provide improved performance on healthcare-related tasks such as medical question answering, clinical reasoning, and health information summarization. The model requires a GPU with at least 24GB of VRAM for quantized inference. Its domain specialization makes it notably more capable than general models on clinical benchmarks, though it should not be used as a substitute for professional medical advice. Released under the Gemma license.
S1 Mini
superwhisper · 752M · runs from 0.7 GB
S1 Mini is a 752M-parameter open language model from superwhisper. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron Nano 9B v2
NVIDIA · 8.9B · runs from 3.5 GB
NVIDIA Nemotron Nano 9B v2 is a compact yet capable chat model from NVIDIA, packing 8.9 billion parameters into a size that runs comfortably on a wide range of consumer GPUs. Built on NVIDIA's Nemotron architecture, it delivers strong instruction-following and conversational performance while keeping VRAM requirements modest. This second-generation Nano model reflects NVIDIA's push to make high-quality language models accessible on local hardware. It's an excellent starting point for users who want a responsive, general-purpose assistant without needing top-tier GPU memory.
Qwen3 4B Thinking 2507
Alibaba · 4.0B · runs from 2.2 GB
Qwen3 4B Thinking 2507 is the reasoning-optimized variant of Alibaba's compact 4-billion-parameter Qwen3 model, released in the July 2025 update cycle. Despite its small size, this thinking variant is tuned to produce chain-of-thought reasoning and step-by-step problem solving, making it a surprisingly capable lightweight reasoner. This model is ideal for users who want basic reasoning and analytical capabilities on very modest hardware. It can run on most consumer GPUs and even some CPU-only setups when quantized, providing an accessible entry point for experimenting with reasoning-style models without any significant hardware investment.
LFM2 24B A2B
Liquid AI · 23.8B · runs from 48.2 GB
LFM2 24B A2B is a 23.8B-parameter open language model from Liquid AI in the LFM2 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
LFM2.5 2.6B DSpark
Liquid AI · 328M · runs from 0.5 GB
LFM2.5 2.6B DSpark is a 328M-parameter open language model from Liquid AI in the LFM2.5 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
A.X K2
skt · 691.7B · runs from 297.9 GB
A.X K2 is a 691.7B-parameter open language model from skt. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
DeepSeek V4 Pro 0813
DeepSeek · 1650.5B · runs from 639.9 GB
DeepSeek-V4-Pro-0813 is DeepSeek's official release of DeepSeek-V4-Pro, superseding an earlier preview build with stronger agentic capabilities aimed at production use. It is a large Mixture-of-Experts model at roughly 1.65 trillion total parameters, built on the V4-Pro architecture with a DSpark speculative-decoding module attached, and it supports configurable reasoning effort (low, high, or max) to control how much it deliberates before answering. It ships with a roughly 1 million token context window and is released under the MIT license, so it can be used and modified freely. At this scale, running it locally means multi-GPU server hardware with hundreds of gigabytes of memory even at 4-bit; most users will access it through a hosted endpoint instead.
Gemma 3 4B IT
Google · 4.3B · runs from 2.0 GB
Gemma 3 4B IT is a small, roughly 4.3-billion-parameter model from Google's earlier Gemma 3 generation, tuned for instruction-following and chat. It accepts image input alongside text, so it can handle visual question answering and image-grounded prompts in addition to text-only conversation. At this size, the model is comfortable on modest consumer hardware, running even on integrated graphics at low quantization, making it one of the most accessible vision-capable models for local use. Released on February 20, 2025, Gemma 3 4B IT predates the Gemma 4 family and is distributed under Google's Gemma license terms rather than a standard open-source license. Its compact size and image-input support make it a practical choice for lightweight, on-device multimodal applications.
Bielik 11B V3.0 Instruct
speakleash · 11.2B · runs from 24.6 GB
Bielik 11B V3.0 Instruct is a 11.2B-parameter open language model from speakleash. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Hermes 3 Llama 3.1 70B
Nous Research · 70.6B · runs from 20.4 GB
Hermes 3 Llama 3.1 70B is a 70.6B-parameter open language model from Nous Research in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
QwQ 32B
Alibaba · 32.8B · runs from 14.8 GB
QwQ 32B is a 32-billion parameter reasoning-focused model from Alibaba Cloud's Qwen family. Unlike standard chat models, QwQ is specifically optimized for step-by-step logical reasoning, complex problem solving, and mathematical tasks. It employs extended chain-of-thought processing, generating detailed internal reasoning before producing final answers, which significantly improves accuracy on challenging analytical problems. The model requires a GPU with at least 24GB of VRAM for quantized inference and delivers reasoning performance competitive with much larger models. It is particularly well suited for users who need strong analytical capabilities for math, science, coding logic, and multi-step problem solving. Released under the Apache 2.0 license.