All LLM Models
Browse 1214 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Qwen AgentWorld 35B A3B
Alibaba · 34.7B · runs from 9.9 GB
18.7K
722Qwen AgentWorld 35B A3B is Alibaba's 35-billion-parameter mixture-of-experts model, with about 3 billion parameters active per token (the A3B in its name). Unlike a typical chat model, it is a language world model: given an agent's action and history, it predicts what the environment does next, covering domains such as terminal, web, Android, and software engineering. Only the active experts run per token, so inference stays fast even though all the weights must fit in memory. At this size, local inference calls for quantization and a single high-end consumer GPU. The model supports a 262K token context window and is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in June 2026, it is built on a Qwen3.5-35B-A3B base and is meant for simulating agents, not direct conversation.
Ternary Bonsai 8B Unpacked
prism-ml · 8.2B · runs from 4.1 GB
Ternary Bonsai 8B Unpacked is a 8.2B-parameter open language model from prism-ml. It supports a context window of up to 65,536 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Hy MT2 1.8B
Tencent · 2.0B · runs from 1.1 GB
Hy MT2 1.8B is a 2-billion-parameter translation model from Tencent's Hunyuan team, part of a three-size Hy-MT2 family (1.8B, 7B, and a 30B mixture-of-experts variant) built specifically for machine translation rather than general chat. It supports translation across roughly three dozen languages and follows translation-specific instructions, such as adjusting tone or terminology. It runs on modest consumer hardware, including laptops, and Tencent has also released extreme-quantized versions shrunk to a few hundred megabytes for on-device use. It supports a 262K token context window, letting it translate long documents in a single pass. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in May 2026, it is the smallest member of a translation-focused family Tencent positions as a self-hostable alternative to commercial translation APIs.
Granite 4.2 8B
IBM · 8.8B · runs from 3.0 GB
Granite 4.2 8B is a 8.8B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
KAT Coder V2.5 Dev
Kwaipilot · 34.7B · runs from 9.9 GB
KAT Coder V2.5 Dev is a 34.7B-parameter open language model from Kwaipilot. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mistral 7B v0.3
Mistral AI · 7.2B · runs from 3.6 GB
Mistral 7B v0.3 is a 7.2B-parameter open language model from Mistral AI in the Mistral family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 VL 8B Instruct
Alibaba · 8.8B · runs from 3.0 GB
Qwen3 VL 8B Instruct is Alibaba's 8.8-billion-parameter vision-language model in the Qwen3-VL lineup, built to process images and text together in one conversation. Beyond image description and visual question answering, it is tuned as a visual agent that can read GUI screenshots and reason about on-screen elements, useful for document analysis and early automation tasks. At this size, local inference is practical on a single mainstream-to-high-end consumer GPU once quantized. The model supports a 262,144 token context window, enough for long documents or extended chat histories. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2025 alongside 2B and 32B siblings, Qwen3-VL adds video understanding with fine-grained event indexing beyond earlier Qwen vision-language releases.
Ornith 1.0 35B
Deep Reinforce · 35.1B · runs from 10.0 GB
Ornith 1.0 35B, from Deep Reinforce, is a 35-billion-parameter model built on a mixture-of-experts architecture, meaning only a subset of its parameters activate for any given token even though the full model must be held in memory. Because the full expert set still needs to fit in memory much like a dense model of the same size, local use is best suited to a single high-end consumer GPU once quantized rather than lower-end hardware. It is tuned for general chat and instruction-following. The model provides a 256K token context window for handling long inputs, and is released under the MIT license, one of the more permissive options in this catalogue. It was published on June 21, 2026, the same day as its smaller sibling, Ornith 1.0 9B.
MiniCPM5 1B Claude Opus Fable5 v2 Thinking
GnLOLot · 1.1B · runs from 0.8 GB
MiniCPM5 1B Claude Opus Fable5 v2 Thinking is a 1.1B-parameter open language model from GnLOLot in the MiniCPM family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
North Mini Code 1.0
Cohere · 30.5B · runs from 8.8 GB
North Mini Code 1.0 is Cohere's first open-weight coding model, a mixture-of-experts design with about 30.5 billion total parameters and roughly 3.3 billion active per token. It is tuned for agentic coding: tool calling, terminal workflows, and software-engineering tasks rather than general chat. Only the active parameters compute per token, so it decodes quickly for its size, though all weights must fit in memory; it runs on a single high-end consumer GPU once quantized. The model supports a 500,000 token context window, useful for large codebases or long agent transcripts. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in June 2026 as the debut release in Cohere's North family, it is designed to run on a single H100-class GPU in FP8.
Llama 3.3 70B Instruct
Meta · 70.6B · runs from 21.3 GB
Meta Llama 3.3 70B Instruct is a 70-billion parameter large language model from Meta, released as part of the Llama 3.3 generation. It is an instruction-tuned model optimized for dialogue and chat use cases, offering strong performance across reasoning, coding, and multilingual tasks. Llama 3.3 70B delivers quality competitive with much larger models while remaining feasible to run on high-end consumer or workstation GPUs with sufficient VRAM. The model uses a grouped-query attention architecture with a 128K token context window and was trained on a massive multilingual corpus. It is released under the Llama 3.3 Community License, making it one of the most capable openly available models for local inference.
Apodex 1.1 Mini
apodex · 36.0B · runs from 12.5 GB
Apodex 1.1 Mini is a 36.0B-parameter open language model from apodex. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 2 9B IT
Google · 9.2B · runs from 3.0 GB
Google Gemma 2 9B IT is a 9.2-billion parameter instruction-tuned model from Google's Gemma 2 series. It is a text-only chat model optimized for conversational tasks, instruction following, and general-purpose assistance. At release, it was recognized for delivering unusually strong performance relative to its parameter count. The model runs efficiently on consumer GPUs with 8-12GB of VRAM in quantized formats, making it accessible on mainstream hardware. It is a popular choice for local inference among users who want strong quality without the VRAM demands of larger models. Released under the Gemma license.
Phi 4 Mini Instruct
Microsoft · 3.8B · runs from 1.9 GB
Microsoft Phi 4 Mini Instruct is a 3.8-billion parameter instruction-tuned model from Microsoft Research's Phi 4 family. It applies the Phi series' data-centric training philosophy to a compact model, delivering strong performance in coding, reasoning, and chat tasks relative to its small footprint. The model runs on consumer GPUs with as little as 4-6GB of VRAM when quantized, making it accessible on mainstream and even entry-level hardware. Released under the MIT license.
Mistral 7B Instruct v0.3
Mistral AI · 7.2B · runs from 2.7 GB
Mistral 7B Instruct v0.3 is the latest instruction-tuned release of Mistral AI's original 7-billion-parameter model, delivering meaningful improvements in instruction following, function calling, and multilingual support over its predecessors. With an extended 32K-token vocabulary and refined chat capabilities, v0.3 remains one of the most capable sub-10B models available. At 7.2 billion parameters it sits comfortably in the sweet spot for local inference, running well on GPUs with 6–8 GB of VRAM at full precision and even on 4 GB cards with 4-bit quantization. It is an excellent default choice for anyone getting started with local LLMs who wants strong conversational performance without heavy hardware.
Qwen2.5 14B
Alibaba · 14.8B · runs from 6.8 GB
Qwen2.5 14B is a 14.8B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Jan v3 4B Base Instruct
janhq · 4.4B · runs from 2.4 GB
Jan v3 4B Base Instruct is a 4.4B-parameter open language model from janhq. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 31B IT Scotoma 2
ReadyArt · 31.3B · runs from 13.7 GB
Gemma 4 31B IT Scotoma 2 is a 31.3B-parameter open language model from ReadyArt in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 0.5B Instruct
Alibaba · 494M · runs from 0.5 GB
Qwen2.5 0.5B Instruct is the smallest instruction-tuned model in Alibaba Cloud's Qwen 2.5 family, with just 494 million parameters. It is designed for ultra-lightweight deployment scenarios where minimal hardware resources are available, running comfortably on virtually any modern GPU or even CPU-only configurations. Despite its tiny footprint, the model supports a 128K token context window and can handle basic chat, simple summarization, and lightweight instruction following. It is primarily useful for edge deployment, experimentation, and prototyping where model size is a critical constraint. Released under the Apache 2.0 license.
Qwen3 Next 80B A3B Instruct
Alibaba · 81.3B · runs from 22.8 GB
Qwen3 Next 80B A3B Instruct is a Mixture of Experts (MoE) model from Alibaba Cloud's Qwen 3 series, with approximately 81.3 billion total parameters and around 3 billion active parameters per forward pass. This extreme ratio between total and active parameters allows the model to encode extensive knowledge across its expert layers while maintaining very fast per-token inference, making it an unusually efficient design for its capability level. The model is instruction-tuned for general-purpose chat and requires VRAM proportional to its full 80B parameter count for weight loading, typically needing high-VRAM GPUs or quantized multi-GPU setups. Its low active parameter count results in fast generation speeds despite the large total model size. Released under the Apache 2.0 license.
Gemma 2 2B IT
Google · 2.6B · runs from 0.9 GB
Google Gemma 2 2B IT is a 2-billion parameter instruction-tuned model from Google's Gemma 2 family, the smallest variant in the Gemma 2 series. It is designed for efficient local inference on resource-constrained hardware, handling basic conversational tasks and simple instruction following at minimal compute cost. The model can run on GPUs with as little as 4GB of VRAM when quantized, and even on CPU-only setups. Released under the Gemma license.
Agents A1 4B
InternScience · 4.5B · runs from 2 GB
Agents A1 4B is a 4.5B-parameter open language model from InternScience in the Agents-A1 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mellum2 12B A2.5B Instruct
JetBrains · 12.1B · runs from 4.0 GB
Mellum2 12B A2.5B Instruct is JetBrains' 12-billion-parameter mixture-of-experts coding model, with about 2.4 billion parameters active per token across 64 experts. Because only a fraction of the weights are computed for each token, it runs noticeably faster than a dense model of similar size, while all the experts still need to fit in memory. Unlike JetBrains' earlier Mellum, a narrow fill-in-the-middle completion model, Mellum2 is a full coding assistant that can generate and edit code, call tools, hold multi-turn conversations, and reason through problems. Its modest active-parameter count makes it practical to run on a single mainstream consumer GPU once quantized. The model supports a 131K token context window and is released under the Apache 2.0 license, allowing unrestricted commercial and research use. It was published in May 2026.
Ling 3.0 Tiny
Inclusion AI · 7.9B · runs from 2.8 GB
Ling 3.0 Tiny is InclusionAI's smallest mixture-of-experts model in Ant Group's Ling 3.0 family, totaling roughly 7.9 billion parameters with about 1.4 billion active per token. It shares the family's hybrid-linear-attention MoE architecture, also used at far larger scale in Ling-3.0-Flash, scaled down here for lightweight chat and agent use. Only the active parameters compute per token, so it responds quickly while still needing its full parameter set in memory; under 8 billion total, it runs on a mainstream consumer GPU once quantized. The model provides a 131K token context window for extended conversations or agent traces. It is released under the MIT license, allowing unrestricted commercial and research use. Published in August 2026, it extends Ant Group's Ling 3.0 lineup, built for efficient agent workloads.
Yi Coder 9B Chat
01.AI · 8.8B · runs from 3.1 GB
Yi Coder 9B Chat is a 8.8B-parameter open language model from 01.AI in the Yi family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 3.5 Mini Instruct
Microsoft · 3.8B · runs from 2.3 GB
Phi 3.5 Mini Instruct is a 3.8B-parameter open language model from Microsoft in the Phi 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.6 35B A3B Claude 4.6 Opus Reasoning Distilled
hesamation · 36.0B · runs from 15.7 GB
Qwen3.6 35B A3B Claude 4.6 Opus Reasoning Distilled is a 36.0B-parameter open language model from hesamation in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Meta Llama 3 8B Instruct
Meta · 8.0B · runs from 2.6 GB
Meta Llama 3 8B Instruct is the instruction-tuned version of Meta's Llama 3 8B base model, with 8 billion parameters. It is fine-tuned for dialogue and chat use cases using supervised fine-tuning and RLHF, making it ready for conversational applications out of the box. The model supports an 8K token context window and performs well across coding, reasoning, and general knowledge tasks. Its efficient size makes it one of the most popular models for local inference on consumer hardware. Released under the Meta Llama 3 Community License.
Qwen3 Next 80B A3B Thinking
Alibaba · 81.3B · runs from 22.8 GB
Qwen3 Next 80B A3B Thinking is Alibaba's 81-billion-parameter mixture-of-experts model, with about 3 billion parameters active per token (the A3B in its name), drawn from 512 experts with only 10 routed at a time. Because only the active experts run per token, inference is far cheaper than a dense model this size, though the full weight set still needs to fit in memory. It combines mostly-linear Gated DeltaNet attention with periodic full attention layers, and this Thinking variant always reasons through a visible chain of thought before answering. Running it locally needs multiple GPUs or a large unified-memory machine. It supports a 262K token context window, is released under the Apache 2.0 license, and was published in September 2025 as part of the Qwen3-Next line introducing this hybrid-attention architecture.
S1 Mini
superwhisper · 752M · runs from 0.7 GB
S1 Mini is a 752M-parameter open language model from superwhisper. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.