All LLM Models
Browse 1432 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Model List
Qwen3.8 27B
Alibaba · 27.8B · runs from 17.9 GB at Q4
7.8M
19.3KAlibaba's Qwen3.8 27B is a dense, vision-capable model with around 28 billion parameters, continuing the Qwen line into its 3.8 release. It can process images alongside text prompts, supporting tasks like visual question answering and image-grounded chat in addition to general-purpose reasoning and coding assistance. At this size, the model fits on a single high-end consumer GPU once quantized, putting it within reach of enthusiast local setups rather than requiring server-class hardware. It ships with a 256K token context window, enough for long-form documents or extended multi-turn sessions, and is distributed under the Apache 2.0 license. Released in August 2026, it represents one of the more recent entries in Alibaba's Qwen 3.8 family of open-weight models.
Qwen3.6 35B A3B
Alibaba · 36.0B · runs from 22.0 GB at Q4
Qwen3.6 35B A3B pairs a mixture-of-experts design with roughly 35 billion total parameters, only about 3 billion of which are active per token. That small active footprint keeps generation speed close to a 3B-class dense model, even though the full parameter set must still be held in memory, similar to a dense model of that size, meaning local use still calls for a single high-end consumer GPU once quantized. The model also accepts image input alongside text, extending it to visual question answering and image-grounded chat. It supports a 256K token context window for long documents and extended conversations, and is released under the Apache 2.0 license. Published in April 2026, it gives Qwen 3.6 users a faster-inference alternative to the dense Qwen3.6 27B model released the same generation.
Gemma 4 26B A4B IT
Google · 25.8B · runs from 16.5 GB at Q4
Gemma 4 26B A4B IT applies a mixture-of-experts design to Google's Gemma 4 family, with roughly 26 billion total parameters but only about 4 billion active for any given token. That active-parameter count is what determines inference speed, so despite its total size the model can respond about as quickly as a much smaller dense model, though the full parameter set still needs to be held in memory, putting local use in single high-end consumer GPU territory once quantized. It also accepts image input alongside text, making it usable for multimodal chat and visual question answering. The model offers a 256K token context window, suited to long documents or extended conversations, and is released under the Apache 2.0 license for unrestricted use. Released alongside the dense 31B variant, it gives developers a faster-inference option within the same Gemma 4 generation.
Qwen3.5 9B
Alibaba · 9.7B · runs from 6.6 GB at Q4
Qwen3.5-9B is Alibaba's dense 9-billion-parameter language model with a built-in vision encoder, part of the Qwen3.5 generation. This is the post-trained, instruction-tuned release, combining Gated DeltaNet and gated-attention layers for efficient long-context inference, and it can take images alongside text for tasks like visual question answering as well as general chat, coding, and tool use. It offers a 262,144-token native context window, extensible up to roughly 1,010,000 tokens, and is released under the Apache 2.0 license. At around 9.7 billion parameters, 4-bit quantization needs only about 5.6GB of memory, so it fits comfortably on a single mid-range consumer GPU.
Gemma 4 E4B IT
Google · 8.0B · runs from 5.6 GB at Q4
Gemma 4 E4B IT packs Google's Gemma 4 architecture into a compact, roughly 8-billion-parameter footprint, positioned as the mid-sized option in Gemma 4's efficiency-focused E-series alongside the smaller E2B variant. It is tuned for chat and instruction-following rather than multimodal input, focusing on dialogue quality within a lightweight package. Parameter counts in this range are well suited to local inference on a single consumer GPU, even at higher precision, and comfortably so once quantized. The model provides a 128K token context window, sufficient for most chat and document-assistance use cases without needing the largest Gemma 4 variants. It is released under the Apache 2.0 license, and was published on March 2, 2026, shortly before the larger dense and mixture-of-experts Gemma 4 models that followed.
Qwen3 8B
Alibaba · 8.2B · runs from 7.6 GB at Q4
Qwen3 8B is an 8.2-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 3 series. It is a general-purpose chat model that delivers strong performance across reasoning, multilingual understanding, and coding tasks while remaining efficient enough to run on consumer GPUs with 8GB or more of VRAM. Like other Qwen 3 models, it supports hybrid thinking mode for flexible reasoning depth. The model benefits from the improved pretraining data and training methodology of the Qwen 3 generation, offering notable quality gains over Qwen 2.5 at the same parameter count. It is widely supported by inference frameworks including llama.cpp, vLLM, and Ollama. Released under the Apache 2.0 license.
Gemma 4 12B IT
Google · 12.0B · runs from 9.0 GB at Q4
Sitting between Gemma 4's compact E-series and its larger 31B sibling, Gemma 4 12B IT is a 12-billion-parameter instruction-tuned model built for general chat and dialogue use. It does not take image input, focusing instead on text-based reasoning, coding help, and conversational tasks. At this parameter count, the model fits comfortably on a single consumer GPU without necessarily requiring the heaviest quantization, making it a reasonable middle-ground choice for local setups. The model supports a generous 256K token context window, useful for long documents or extended sessions, and is released under the Apache 2.0 license. It arrived in May 2026, after the initial March 2026 releases of the Gemma 4 E-series and 31B dense model, suggesting continued iteration within the family after the launch wave.
Qwen2.5 VL 7B Instruct
Alibaba · 8.3B · runs from 6.2 GB at Q4
Qwen2.5 VL 7B Instruct is Alibaba's 8.3-billion-parameter vision-language model in the Qwen 2.5 lineup, built to handle images and video alongside text in a single pass. It can read documents, charts, and screenshots, describe and reason about visual content, and point out object locations within an image, useful for document understanding, visual question answering, and lightweight visual-agent tasks. At this parameter count, local inference is practical on a single mainstream or high-end consumer GPU once the weights are quantized. The model supports a 128K token context window, enough for long documents or extended multi-turn visual conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and was published in January 2025 as part of Qwen's second-generation vision-language line, later succeeded by Qwen3-VL.
Qwen2.5 7B Instruct
Alibaba · 7.6B · runs from 5.8 GB at Q4
Qwen2.5 7B Instruct is a 7.6-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 2.5 series. It supports a 128K token context window and is fine-tuned for conversational AI, instruction following, and general assistant tasks. Its efficient size makes it well-suited for local deployment on consumer GPUs with 8GB or more of VRAM. The model delivers strong performance for its parameter class across reasoning, multilingual understanding, and coding tasks. It benefits from the improved pretraining data and techniques of the Qwen 2.5 generation. Released under the Apache 2.0 license and widely supported by inference frameworks such as llama.cpp, vLLM, and Ollama.
Gemma 4 31B IT
Google · 31.3B · runs from 21.7 GB at Q4
Gemma 4 31B IT is Google's 31-billion-parameter instruction-tuned model in the Gemma 4 lineup, built to handle both text and image input in a single pass. As a vision-capable model, it can describe, compare, or reason about images alongside written prompts, making it suitable for multimodal chat and document-understanding tasks. At this parameter count, local inference calls for quantization and a fairly capable GPU; it fits on a single high-end consumer or workstation card rather than lower-end hardware. The model supports a 256K token context window, enough for long documents, transcripts, or multi-turn conversations without aggressive truncation. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and reflects Google's continued push toward mid-sized, multimodal open-weight models following the earlier Gemma generations.
Qwen3.6 27B
Alibaba · 27.8B · runs from 17.9 GB at Q4
Released a few months before its Qwen3.8 sibling, Qwen3.6 27B is Alibaba's dense, roughly 28-billion-parameter model from the Qwen 3.6 generation. Vision input is supported alongside text, so the model can handle image-based prompts as well as standard chat, coding, and reasoning tasks. Like other models in this size class, it is best run locally on a single high-end consumer GPU with quantization rather than lower-end hardware. Qwen3.6 27B carries a 256K token context window for handling lengthy inputs, and is released under the Apache 2.0 license, permitting free commercial and research use. It sits alongside the larger Qwen3.6 35B A3B mixture-of-experts model as one of two Qwen 3.6 options with local-deployment potential.
Qwen3.5 4B
Alibaba · 4.7B · runs from 3.4 GB at Q4
Qwen3.5-4B is Alibaba's dense 4-billion-parameter vision-language model from the Qwen3.5 generation, sharing the same hybrid Gated DeltaNet and gated-attention architecture as its larger siblings. This is the post-trained, instruction-tuned release, able to process images together with text and handle general chat, coding assistance, and agentic tool use. It has a 262,144-token native context window, extensible up to roughly 1,010,000 tokens, and is released under the Apache 2.0 license. At around 4.7 billion parameters, 4-bit quantization needs under 3GB of memory, so it runs easily on almost any modern GPU or laptop.
Qwen3 14B
Alibaba · 14.8B · runs from 11.8 GB at Q4
Qwen3 14B is a 14-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 3 series. It occupies a practical middle ground in the Qwen 3 lineup, offering stronger reasoning and generation quality than the 8B variant while remaining manageable on GPUs with 16GB or more of VRAM in quantized formats. The model supports hybrid thinking mode for flexible reasoning depth. Qwen3 14B is well suited for chat, instruction following, coding assistance, and multilingual tasks. It benefits from the generational improvements of Qwen 3 in pretraining data and alignment techniques, delivering performance that competes with larger models from previous generations. Released under the Apache 2.0 license.
Qwen3 4B Instruct 2507
Alibaba · 4.0B · runs from 4.2 GB at Q4
Qwen3 4B Instruct 2507 is a July 2025 refresh of Alibaba's compact 4-billion-parameter chat model from the Qwen3 family. This updated release brings improved instruction following and conversational quality while remaining lightweight enough to run on most modern GPUs and even some higher-end integrated graphics setups. With its modest size, the 4B Instruct 2507 strikes a practical balance between capability and resource efficiency. It is well suited for everyday chat, summarization, and light assistant tasks on consumer hardware, making it one of the more accessible entry points into the Qwen3 lineup.
Qwen3 VL 8B Instruct
Alibaba · 8.8B · runs from 8.0 GB at Q4
Qwen3 VL 8B Instruct is Alibaba's 8.8-billion-parameter vision-language model in the Qwen3-VL lineup, built to process images and text together in one conversation. Beyond image description and visual question answering, it is tuned as a visual agent that can read GUI screenshots and reason about on-screen elements, useful for document analysis and early automation tasks. At this size, local inference is practical on a single mainstream-to-high-end consumer GPU once quantized. The model supports a 262,144 token context window, enough for long documents or extended chat histories. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2025 alongside 2B and 32B siblings, Qwen3-VL adds video understanding with fine-grained event indexing beyond earlier Qwen vision-language releases.
Qwen2.5 32B Instruct
Alibaba · 32.8B · runs from 24.3 GB at Q4
Qwen2.5 32B Instruct is a 32-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 2.5 family. It occupies a practical sweet spot between the 14B and 72B variants, offering strong reasoning and multilingual capabilities while remaining feasible to run on a single high-end consumer GPU with 24GB or more of VRAM at reduced precision. The model supports a 128K token context window and is optimized for conversational use, instruction following, and structured output generation. It is a popular choice for local inference when the 72B model is too demanding but users need more capability than the 14B variant. Released under the Apache 2.0 license.
Qwen2.5 Coder 7B Instruct
Alibaba · 7.6B · runs from 5.8 GB at Q4
Qwen2.5 Coder 7B Instruct is a 7.6-billion parameter code-specialized instruction-tuned model from Alibaba Cloud. It is trained on a large corpus of source code and natural language, fine-tuned for programming assistance tasks such as code generation, completion, debugging, and code explanation. The model supports a 128K token context window and runs efficiently on consumer GPUs with 8GB or more of VRAM. It provides a good balance between coding capability and hardware requirements for developers looking to run a local coding assistant. Released under the Apache 2.0 license.
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP
DavidAU · 9.7B · runs from 6.6 GB at Q4
Qwen3.5 9B The Defiant Fable Uncensored Heretic NEO IMATRIX MAX MTP is a 9.7B-parameter open language model from DavidAU in the Qwen 3.5 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 Coder 30B A3B Instruct
Alibaba · 30.5B · runs from 19.4 GB at Q4
Qwen3 Coder 30B A3B Instruct is a code-specialized Mixture of Experts (MoE) model from Alibaba Cloud's Qwen 3 Coder series, with 30 billion total parameters and approximately 3 billion active parameters per forward pass. The MoE architecture allows it to deliver strong coding performance while keeping per-token compute costs low, making it faster at inference than comparably capable dense models. The model is instruction-tuned for programming assistance, code generation, debugging, and software engineering conversation. It requires VRAM proportional to its total 30B parameter count for loading weights, but benefits from efficient inference throughput due to its low active parameter count. Released under the Apache 2.0 license.
Qwen2.5 3B Instruct
Alibaba · 3.1B · runs from 2.8 GB at Q4
Qwen2.5 3B Instruct is a 3.1-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 2.5 family. It is designed for efficient local inference on consumer hardware, supporting a 128K token context window despite its compact footprint. The model can run on GPUs with as little as 4GB of VRAM when quantized. Despite its small size, Qwen2.5 3B Instruct delivers competitive performance for basic conversational tasks, summarization, and simple instruction following. It is a good option for edge deployment and resource-constrained environments. Released under the Apache 2.0 license.
Ornith 1.5 9B Uncensored
junafinity · 9.4B · runs from 6.5 GB at Q4
Ornith 1.5 9B Uncensored is a 9.4B-parameter open language model from junafinity in the Ornith family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 4B
Alibaba · 4.0B · runs from 4.2 GB at Q4
Qwen3 4B is a compact 4-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 3 family. It is designed for efficient local inference on consumer hardware, supporting chat and general assistant tasks while fitting comfortably on GPUs with 6GB or more of VRAM in quantized formats. The model supports hybrid thinking mode, allowing it to balance reasoning depth and response speed. Despite its small footprint, Qwen3 4B delivers quality competitive with larger models from previous generations, making it a practical choice for lightweight local deployments and resource-constrained environments. Released under the Apache 2.0 license.
Gemma 4 E2B IT
Google · 5.1B · runs from 3.5 GB at Q4
The smallest model discussed here from Google's Gemma 4 E-series, Gemma 4 E2B IT, comes in at roughly 5 billion parameters. It is tuned for chat and instruction-following rather than multimodal tasks, prioritizing a minimal footprint over broad input support. Models at this size are comfortable on typical consumer GPUs, and can even run on integrated graphics when quantized aggressively, making it one of the more accessible options for local deployment. It ships with a 128K token context window, matching its larger E4B sibling, and is released under the Apache 2.0 license for open commercial and research use. Google published it on March 2, 2026, alongside the E4B variant, as the entry point into the Gemma 4 generation.
Qwen2.5 14B Instruct
Alibaba · 14.8B · runs from 12.4 GB at Q4
Qwen2.5 14B Instruct is a 14-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 2.5 series. It supports a 128K token context window and provides a balanced tradeoff between quality and hardware requirements, running well on GPUs with 16GB of VRAM in quantized formats. The model is fine-tuned for chat, instruction following, and general-purpose assistant tasks. It performs well across reasoning, coding, and multilingual benchmarks for its size class, making it a practical option for local deployment when larger models are not feasible. Released under the Apache 2.0 license.
Llama 3.2 1B Instruct
Meta · 1.2B · runs from 2.3 GB at Q4
Meta Llama 3.2 1B Instruct is a 1-billion parameter instruction-tuned model from Meta, the smallest in the Llama 3.2 family. It is designed for ultra-lightweight deployment scenarios where minimal hardware resources are available, supporting a 128K token context window despite its compact size. This model is suitable for basic conversational tasks, text summarization, and simple instruction following. It can run on virtually any modern GPU and even on CPU-only setups with acceptable performance. Released under the Llama 3.2 Community License.
Gemma 4 E4B IT Qat Q4 0 Unquantized
Google · 7.9B · runs from 5.5 GB at Q4
Gemma 4 E4B IT Qat Q4 0 Unquantized is a 7.9B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron 3.5 Lightning 30B A3B BF16
NVIDIA · 31.6B · runs from 19.3 GB at Q4
NVIDIA Nemotron 3.5 Lightning 30B A3B BF16 is a mixture-of-experts chat model in NVIDIA's Nemotron line, with roughly 30 billion total parameters and about 3 billion active per token. That active-parameter count drives inference speed, while the larger total governs memory footprint, and the Lightning name points to a variant tuned for fast responses; weights are published in BF16 precision. The model offers a 256K token context window and is released under a custom license rather than a standard open-source one. With only about 3 billion parameters active per token, it is well suited to running on a single high-end consumer GPU, especially once quantized, despite its larger total parameter count.
Gemma 4 12B IT Qat Q4 0 Unquantized
Google · 12.0B · runs from 9.0 GB at Q4
Gemma 4 12B IT Qat Q4 0 Unquantized is a 12.0B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.8 35B A3B Distill
empero-ai · 35.1B · runs from 21.5 GB at Q4
Qwen3.8 35B A3B Distill is a 35.1B-parameter open language model from empero-ai in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 VL 4B Instruct
Alibaba · 4.4B · runs from 4.5 GB at Q4
Qwen3 VL 4B Instruct is Alibaba's 4.4-billion-parameter vision-language model from the Qwen3-VL family, built to read and reason over images and video alongside text in one conversation. It suits multimodal chat, document and chart understanding, and simple visual-agent tasks like describing screenshots or pulling structured data out of pictures. At this size the model runs comfortably on a mainstream consumer GPU once quantized, and is light enough for many laptops. The model supports a 262K token context window, giving room for long documents or extended visual conversations. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in October 2025 as part of the Qwen3-VL generation, built for deeper visual perception and stronger agentic behavior than the earlier Qwen2-VL series.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.