All LLM Models
Browse 58 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Model List
Gemma 4 26B A4B IT
Google · 25.8B · runs from 16.5 GB at Q4
5.7M
1.7KGemma 4 26B A4B IT applies a mixture-of-experts design to Google's Gemma 4 family, with roughly 26 billion total parameters but only about 4 billion active for any given token. That active-parameter count is what determines inference speed, so despite its total size the model can respond about as quickly as a much smaller dense model, though the full parameter set still needs to be held in memory, putting local use in single high-end consumer GPU territory once quantized. It also accepts image input alongside text, making it usable for multimodal chat and visual question answering. The model offers a 256K token context window, suited to long documents or extended conversations, and is released under the Apache 2.0 license for unrestricted use. Released alongside the dense 31B variant, it gives developers a faster-inference option within the same Gemma 4 generation.
Gemma 4 E4B IT
Google · 8.0B · runs from 5.6 GB at Q4
Gemma 4 E4B IT packs Google's Gemma 4 architecture into a compact, roughly 8-billion-parameter footprint, positioned as the mid-sized option in Gemma 4's efficiency-focused E-series alongside the smaller E2B variant. It is tuned for chat and instruction-following rather than multimodal input, focusing on dialogue quality within a lightweight package. Parameter counts in this range are well suited to local inference on a single consumer GPU, even at higher precision, and comfortably so once quantized. The model provides a 128K token context window, sufficient for most chat and document-assistance use cases without needing the largest Gemma 4 variants. It is released under the Apache 2.0 license, and was published on March 2, 2026, shortly before the larger dense and mixture-of-experts Gemma 4 models that followed.
Gemma 4 12B IT
Google · 12.0B · runs from 9.0 GB at Q4
Sitting between Gemma 4's compact E-series and its larger 31B sibling, Gemma 4 12B IT is a 12-billion-parameter instruction-tuned model built for general chat and dialogue use. It does not take image input, focusing instead on text-based reasoning, coding help, and conversational tasks. At this parameter count, the model fits comfortably on a single consumer GPU without necessarily requiring the heaviest quantization, making it a reasonable middle-ground choice for local setups. The model supports a generous 256K token context window, useful for long documents or extended sessions, and is released under the Apache 2.0 license. It arrived in May 2026, after the initial March 2026 releases of the Gemma 4 E-series and 31B dense model, suggesting continued iteration within the family after the launch wave.
Gemma 4 31B IT
Google · 31.3B · runs from 21.7 GB at Q4
Gemma 4 31B IT is Google's 31-billion-parameter instruction-tuned model in the Gemma 4 lineup, built to handle both text and image input in a single pass. As a vision-capable model, it can describe, compare, or reason about images alongside written prompts, making it suitable for multimodal chat and document-understanding tasks. At this parameter count, local inference calls for quantization and a fairly capable GPU; it fits on a single high-end consumer or workstation card rather than lower-end hardware. The model supports a 256K token context window, enough for long documents, transcripts, or multi-turn conversations without aggressive truncation. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and reflects Google's continued push toward mid-sized, multimodal open-weight models following the earlier Gemma generations.
Gemma 4 E2B IT
Google · 5.1B · runs from 3.5 GB at Q4
The smallest model discussed here from Google's Gemma 4 E-series, Gemma 4 E2B IT, comes in at roughly 5 billion parameters. It is tuned for chat and instruction-following rather than multimodal tasks, prioritizing a minimal footprint over broad input support. Models at this size are comfortable on typical consumer GPUs, and can even run on integrated graphics when quantized aggressively, making it one of the more accessible options for local deployment. It ships with a 128K token context window, matching its larger E4B sibling, and is released under the Apache 2.0 license for open commercial and research use. Google published it on March 2, 2026, alongside the E4B variant, as the entry point into the Gemma 4 generation.
Gemma 4 E4B IT Qat Q4 0 Unquantized
Google · 7.9B · runs from 5.5 GB at Q4
Gemma 4 E4B IT Qat Q4 0 Unquantized is a 7.9B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 12B IT Qat Q4 0 Unquantized
Google · 12.0B · runs from 9.0 GB at Q4
Gemma 4 12B IT Qat Q4 0 Unquantized is a 12.0B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 3 27B IT
Google · 27.4B · runs from 21.7 GB at Q4
Google Gemma 3 27B IT is a 27.4-billion parameter multimodal instruction-tuned model from Google's Gemma 3 family. It supports both text and image inputs, making it one of the most capable openly available vision-language models for local inference. The model handles conversational AI, visual question answering, image description, and complex reasoning tasks across modalities. Gemma 3 27B IT requires a GPU with at least 24GB of VRAM for quantized inference, placing it within reach of high-end consumer cards like the RTX 4090. It uses a dense Transformer architecture with a large context window and benefits from Google's extensive pretraining pipeline. Released under the Gemma license.
Gemma 2 9B IT
Google · 9.2B · runs from 7.3 GB at Q4
Google Gemma 2 9B IT is a 9.2-billion parameter instruction-tuned model from Google's Gemma 2 series. It is a text-only chat model optimized for conversational tasks, instruction following, and general-purpose assistance. At release, it was recognized for delivering unusually strong performance relative to its parameter count. The model runs efficiently on consumer GPUs with 8-12GB of VRAM in quantized formats, making it accessible on mainstream hardware. It is a popular choice for local inference among users who want strong quality without the VRAM demands of larger models. Released under the Gemma license.
Gemma 3 1B IT
Google · 1000M · runs from 2.1 GB at Q4
Google Gemma 3 1B IT is a 1-billion parameter instruction-tuned model from Google's Gemma 3 family. It is an ultra-compact text-only chat model designed for deployment on minimal hardware, including low-VRAM GPUs and edge devices. The model handles basic conversational tasks, simple instruction following, and lightweight text generation. It can run on virtually any modern GPU and even on CPU-only setups with acceptable latency. Released under the Gemma license.
Gemma 3 4B IT
Google · 4.3B · runs from 5 GB at Q4
Gemma 3 4B IT is a small, roughly 4.3-billion-parameter model from Google's earlier Gemma 3 generation, tuned for instruction-following and chat. It accepts image input alongside text, so it can handle visual question answering and image-grounded prompts in addition to text-only conversation. At this size, the model is comfortable on modest consumer hardware, running even on integrated graphics at low quantization, making it one of the most accessible vision-capable models for local use. Released on February 20, 2025, Gemma 3 4B IT predates the Gemma 4 family and is distributed under Google's Gemma license terms rather than a standard open-source license. Its compact size and image-input support make it a practical choice for lightweight, on-device multimodal applications.
Gemma 4 26B A4B IT Qat Q4 0 Unquantized
Google · 25.8B · runs from 16.5 GB at Q4
Gemma 4 26B A4B IT Qat Q4 0 Unquantized is a 25.8B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 3 12B IT
Google · 12.2B · runs from 10.9 GB at Q4
Google Gemma 3 12B IT is a 12-billion parameter multimodal instruction-tuned model from Google's Gemma 3 series. It supports both text and image inputs, offering vision-language capabilities at a more accessible size point than the 27B variant. Gemma 3 12B IT runs on consumer GPUs with 12-16GB of VRAM in quantized formats, making it a practical choice for local multimodal inference without requiring top-tier hardware. Released under the Gemma license.
Gemma 4 E2B IT Qat Q4 0 Unquantized
Google · 5.1B · runs from 3.5 GB at Q4
Gemma 4 E2B IT Qat Q4 0 Unquantized is a 5.1B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 3 270M IT
Google · 268M · runs from 1.2 GB at Q4
Google Gemma 3 270M IT is a 270-million parameter instruction-tuned model from Google's Gemma 3 family, an experimental release pushing the boundaries of how small an effective chat model can be. The model runs on virtually any hardware, including entry-level GPUs and CPU-only setups, making it useful for experimentation, education, and exploring the limits of small-scale language modeling. Released under the Gemma license.
Translategemma 4B IT
Google · 4.3B · runs from 5 GB at Q4
TranslateGemma 4B IT serves as Google's roughly 5-billion-parameter instruction-tuned model designed for translation, built upon the Gemma 3 vision-language architecture and capable of accepting both text and images. As its name suggests, it focuses on machine translation between languages instead of general chat. The listed size accounts for the vision components as well. At this scale, once quantized, it fits comfortably on a single modest consumer GPU and can even run on a laptop. It is distributed under the Gemma Terms of Use, meaning access requires acknowledging the license on Hugging Face. Published in January 2026, this model belongs to Google's Gemma family of open models and comes from the Gemma 3 line.
Gemma 3n E4B IT
Google · 7.8B · runs from 7.7 GB at Q4
Gemma 3n E4B IT is the instruction-tuned variant of Google's Gemma 3n E4B, a multimodal model built for on-device use on phones, laptops, and tablets that accepts text, image, audio, and video input and can perform automatic speech recognition and speech translation alongside text chat. It uses Google's MatFormer (Matryoshka Transformer) architecture, which nests a smaller sub-model inside the full network so the same checkpoint can run at reduced effective capacity, and Per-Layer Embedding, which caches embedding parameters to fast local storage instead of holding them all in memory. The "E4B" designation refers to an effective parameter count of around 4 billion at inference, even though the checkpoint's total parameters are larger; either way it is light enough for a single consumer GPU or even a high-end phone. Context length is 32,768 tokens. It is released under Google's Gemma Terms of Use, a custom license permitting broad commercial and research use alongside a prohibited-use policy, and was published in June 2025.
Gemma 2 2B IT
Google · 2.6B · runs from 2.5 GB at Q4
Google Gemma 2 2B IT is a 2-billion parameter instruction-tuned model from Google's Gemma 2 family, the smallest variant in the Gemma 2 series. It is designed for efficient local inference on resource-constrained hardware, handling basic conversational tasks and simple instruction following at minimal compute cost. The model can run on GPUs with as little as 4GB of VRAM when quantized, and even on CPU-only setups. Released under the Gemma license.
Gemma 3n E2B IT
Google · 5.4B · runs from 5.9 GB at Q4
Gemma 3n E2B is the smaller of Google's two Gemma 3n instruction-tuned models, designed for phones, laptops and other on-device use. "E2B" stands for an effective size of about 2 billion parameters: the raw checkpoint holds around 5.4 billion, but techniques such as per-layer embeddings let it run with a memory footprint closer to a 2B model. It accepts text, image and audio input and generates text. The model has a 32K-token context window and is released under Google's Gemma terms of use. At 4-bit quantization it needs only a few gigabytes of memory, so it runs on almost any modern GPU, laptop or recent phone.
Diffusiongemma 26B A4B IT
Google · 25.8B · runs from 16.5 GB at Q4
Diffusiongemma 26B A4B IT takes a different approach from the rest of the Gemma 4 family: rather than generating tokens autoregressively, it uses a block-diffusion architecture. It still follows a mixture-of-experts design, with roughly 26 billion total parameters and about 4 billion active per token. The active-parameter figure is the more relevant one for inference cost, while the total parameter count determines memory needs, putting local use in single high-end consumer GPU territory once quantized. It also accepts image input alongside text for multimodal use. The model offers a 256K token context window and is released under the Apache 2.0 license. Published on June 9, 2026, it represents Google's exploration of diffusion-based language modeling within the Gemma 4 generation, alongside the more conventional autoregressive Gemma 4 variants also released that year.
Medgemma 4B IT
Google · 4.3B · runs from 5 GB at Q4
MedGemma 4B is Google's 4.3-billion-parameter instruction-tuned vision-language model, built on Gemma 3 and further trained for medical text and image understanding. It adds a medical image encoder covering chest X-rays, dermatology photos, histopathology slides, and fundus images, and can generate findings and answer questions about medical images. It is a foundation for developers building healthcare applications, not a validated diagnostic tool, and needs further evaluation before clinical use. It runs on a single consumer GPU once quantized. It inherits Gemma 3's 128,000 token context window. It is released under Google's Health AI Developer Foundations terms, a use-restricted license rather than a fully open one. Published in May 2025, it was Google's first instruction-tuned multimodal MedGemma release, alongside a larger text-only 27B variant.
Gemma 4 E2B IT Qat Mobile Transformers
Google · 2.3B · runs from 2.1 GB at Q4
Gemma 4 E2B IT Qat Mobile Transformers is a 2.3B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 2 27B IT
Google · 27.2B · runs from 19.5 GB at Q4
Google Gemma 2 27B IT is a 27.2-billion parameter instruction-tuned model from Google's Gemma 2 generation. It is a text-only chat model optimized for conversational use, reasoning, and instruction following. Gemma 2 27B IT was one of the strongest openly available models in its size class at release. The model requires a GPU with at least 24GB of VRAM for quantized local inference. It is widely supported by popular inference engines and remains a strong choice for users seeking high-quality local chat without needing 70B-class hardware. Released under the Gemma license.
Gemma 4 26B A4B IT Assistant
Google · 26B · runs from 16.0 GB at Q4
Gemma 4 26B A4B IT Assistant is a 26B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Medgemma 1.5 4B IT
Google · 4.3B · runs from 5 GB at Q4
MedGemma 1.5 4B is Google's 4.3-billion-parameter instruction-tuned vision-language model in the MedGemma line of health-AI foundation models, succeeding the original MedGemma 4B at the same size. It is built for developers creating healthcare applications, covering tasks such as medical image interpretation and clinical text understanding, and is not a validated diagnostic tool: outputs need independent verification and further evaluation before any clinical use. At this size it runs on a single consumer GPU once quantized. It is released under Google's Health AI Developer Foundations terms, a use-restricted license that requires accepting specific health-AI conditions on Hugging Face rather than a fully open one. Published in January 2026, it is the second 4B release in the MedGemma series.
Medgemma 27B IT
Google · 27.4B · runs from 21.7 GB at Q4
MedGemma 27B IT is Google's instruction-tuned medical model built from the base Gemma 3 27B architecture. Hugging Face metadata marks it for tasks involving medical text and imaging, including radiology report generation, chest X-ray analysis, pathology review, dermatology checks, and fundus image interpretation; it accepts both textual and visual inputs. Designed as a foundation for health-related tools rather than a general-purpose assistant, the model contains roughly 28.8 billion parameters when counting its vision encoder. Once quantized, it runs on a single high-memory GPU, though consumer cards require heavy compression to function. Like Gemma 3 27B, this variant supports a context window of up to 128K tokens. Its release falls under the Health AI Developer Foundations terms instead of a standard open-source license, meaning users must acknowledge these conditions on Hugging Face before access is granted. Published in July 2025, it succeeds Gemma 3 and includes smaller MedGemma siblings as part of its family.
Gemma 4 E4B IT Assistant
Google · 4B · runs from 2.7 GB at Q4
Gemma 4 E4B IT Assistant is a 4B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 26B A4B
Google · 25.8B · runs from 16.5 GB at Q4
Gemma 4 26B A4B is Google DeepMind's multimodal mixture-of-experts model—the pretrained checkpoint that lacks the -it suffix. The card lists 25.2 billion total parameters and 3.8 billion active ones, with 8 of 128 experts in use plus one shared expert. It combines sliding-window and global attention and takes text and image input. Described as capable reasoners, the Gemma 4 models support configurable thinking modes, native function calling, and system-prompt instructions; furthermore, the card notes that the A4B model runs much faster than its total size would suggest. All weights must stay in memory, yet it fits on a single 24 GB-class consumer GPU once quantized. The documentation provides a 256K-token (262,144-token) context window. Released under the Apache 2.0 license, it allows unrestricted commercial and research applications. Published in March 2026, this model joins a family including E2B, E4B, as well as dense siblings of 12B and 31B parameters.
Gemma 4 E2B
Google · 5.1B · runs from 3.5 GB at Q4
Gemma 4 E2B is Google DeepMind's smallest model in the Gemma 4 family, a dense architecture with roughly 5.1 billion total parameters, of which Google describes about 2.3 billion as its effective footprint at inference. This is the pretrained base checkpoint, not an instruction-tuned chat model, meant as a starting point for fine-tuning. The Gemma 4 family is multimodal — text, image, and at this size natively audio — and E2B targets efficient on-device execution on phones and laptops. It is easy to run locally, even on modest hardware once quantized. It supports a 131,072 token context window. It carries Google's Gemma 4 license terms, published as Apache 2.0 on Hugging Face with additional Gemma-specific usage terms linked from the card. Published in March 2026, it is the smallest of five sizes in the Gemma 4 lineup, aimed at mobile and edge deployment.
Gemma 4 E4B
Google · 8.0B · runs from 5.6 GB at Q4
Gemma 4 E4B is Google DeepMind's second-smallest model in the Gemma 4 family, a dense architecture with roughly 8 billion total parameters, of which Google describes about 4.5 billion as its effective footprint at inference. This is the pretrained base checkpoint rather than an instruction-tuned model, meant for fine-tuning rather than direct chat use. Like the rest of the family it is multimodal — text, image, and natively audio at this size — targeting efficient on-device deployment on laptops and higher-end phones. It runs comfortably on a single consumer GPU, or on-device once quantized. It supports a 131,072 token context window. It carries Google's Gemma 4 license terms, published as Apache 2.0 on Hugging Face with additional Gemma-specific usage terms linked from the card. Published in March 2026, it sits between the E2B on-device model and the larger 12B, 26B-A4B, and 31B tiers.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.