All LLM Models

Browse 1242 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Granite 4.1 8B

IBM · 8.8B · runs from 4.4 GB

Granite-4.1-8B is IBM's 8-billion-parameter long-context instruct model, fine-tuned from Granite-4.1-8B-Base on a mix of permissively licensed open datasets and internally collected synthetic data. An improved post-training pipeline of supervised fine-tuning and reinforcement learning gives it stronger tool calling, instruction following, and chat behavior, and IBM positions it for business assistants, retrieval-augmented generation, summarization, classification, and code tasks including fill-in-the-middle completions, across a dozen supported languages. It offers a 131,072-token context window and is released under the Apache 2.0 license. At around 8.8 billion parameters, 4-bit quantization needs roughly 5GB of memory, so it fits comfortably on an 8-12GB consumer GPU.

CAJAL 4B

Agnuxo · 4B · runs from 1.9 GB

CAJAL 4B is a 4B-parameter open language model from Agnuxo. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

LLaMmlein 1B Prerelease

LSX-UniWue · 1.1B · runs from 0.8 GB

LLaMmlein 1B Prerelease is a 1.1B-parameter open language model from LSX-UniWue. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Wildguard

Allen AI · 7.2B · runs from 3.4 GB

Wildguard is a 7.2B-parameter open language model from Allen AI. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

VLM2Vec Full

TIGER-Lab · 4.1B · runs from 9.4 GB

VLM2Vec Full is a 4.1B-parameter open language model from TIGER-Lab. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 2 2B

Google · 2.6B · runs from 1.2 GB

Google Gemma 2 2B is a 2-billion parameter base (pretrained) model from Google's Gemma 2 family. As a base model, it is not instruction-tuned and is intended for fine-tuning, research, and custom downstream applications. Its compact size makes it suitable for experimentation, rapid prototyping, and domain-specific fine-tuning on consumer hardware with minimal VRAM. Released under the Gemma license.

North Micro Vision Instruct

Cohere · 2.5B · runs from 5.5 GB

North Micro Vision Instruct is Cohere's 2.5-billion-parameter vision-language model, built to handle native-resolution images alongside text in multi-turn conversations. It pairs a custom vision encoder with a small in-house language model, tuned for document understanding, chart and table reading, spatial reasoning, and visual grounding across a dozen-plus languages. Its small size suits local deployment on modest consumer GPUs once quantized. The model supports a very large 500,000 token context window, well beyond most models its size, useful for many-page documents or long image-heavy conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in August 2026 as part of Cohere's North family, it accepts inputs at resolutions up to roughly an A4 page at 200 dpi, aimed at document-heavy enterprise use.

Gpt2 Small Dutch

GroNLP · 129M · runs from 0.1 GB

Gpt2 Small Dutch is a 129M-parameter open language model from GroNLP. It supports a context window of up to 1,024 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MiMo 7B Base

Xiaomi · 7.8B · runs from 3.9 GB

MiMo 7B Base is a 7.8B-parameter open language model from Xiaomi. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen2.5 1.5B Quantized.w8a8

RedHatAI · 1.8B · runs from 1.1 GB

Qwen2.5 1.5B Quantized.w8a8 is a 1.8B-parameter open language model from RedHatAI in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 2 13B Chat HF

Meta · 13.0B · runs from 6.1 GB

Meta Llama 2 13B Chat is a 13-billion parameter instruction-tuned model from Meta's Llama 2 family, fine-tuned for dialogue and chat applications. It offers improved reasoning and generation quality over the 7B variant while maintaining manageable hardware requirements with a 4K token context window. The model was fine-tuned using supervised fine-tuning and RLHF. It can run on consumer GPUs with 16GB or more of VRAM at reduced precision. Released under the Llama 2 Community License.

Gemma 4 12B

Google · 12.0B · runs from 6.1 GB

Gemma 4 12B Unified is a base checkpoint in Google DeepMind's Gemma 4 family, an open-weight multimodal model with roughly 12 billion parameters that takes text, image, and audio input and produces text output. Its "Unified" design skips separate encoders per modality, projecting raw image patches and audio directly into the language model, cutting multimodal latency. As a pretrained release, it's a base for fine-tuning or research rather than direct chat use. It supports a 256K token context window and multilingual pretraining across 140+ languages, and is released under the Apache 2.0 license. At roughly 12 billion parameters, it runs on a single consumer GPU with around 8-12 GB of VRAM once quantized to 4-bit.

Llm Jp 3 150M

llm-jp · 152M · runs from 0.4 GB

Llm Jp 3 150M is a 152M-parameter open language model from llm-jp. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Idefics3 8B Llama3

HuggingFaceM4 · 8.5B · runs from 4.2 GB

Idefics3-8B-Llama3 is Hugging Face's 8.5-billion-parameter open vision-language chat model, accepting arbitrary interleaved sequences of images and text and answering in text: image captioning, visual question answering, multi-image storytelling, or plain text-only chat. It combines a SigLIP-SO400M vision encoder with a Llama-3.1-8B-Instruct language backbone, tiling each image into 364x364 sub-images encoded as 169 visual tokens apiece, which sharply improves OCR and document-understanding scores over the earlier Idefics2. Its post-training is supervised fine-tuning only, without an RLHF stage, so it can give terse answers unless prompted further. At 8.5 billion parameters, it fits on a single consumer GPU once quantized. Context length is 131,072 tokens, inherited from its Llama 3.1 backbone. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in August 2024, succeeding Idefics1 and Idefics2 in the same open multimodal model family.

Moondream3 Preview

moondream · 9.3B · runs from 20.4 GB

Moondream 3 (Preview) is a 9.3-billion-parameter vision-language model from Moondream, built as a sparse Mixture-of-Experts with 64 experts per layer and only 2 billion parameters active per token. It targets visual reasoning such as object detection, counting, pointing, and visual question answering, aiming for accuracy closer to larger closed models while keeping inference light. All expert weights must fit in memory, but the active footprint keeps it usable on a single high-end consumer GPU once quantized. Moondream reports a usable context window of around 32,000 tokens. It ships under a Business Source License 1.1 with an additional use grant: free for personal, research, and most commercial use, but resale as a hosted service is blocked. Published in September 2025 as a preview, it introduces a mixture-of-experts design not present in earlier, dense Moondream models.

Codegen 350M Mono

Salesforce · 350M · runs from 0.8 GB

Codegen 350M Mono is a 350M-parameter open language model from Salesforce. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Ilama 3.2 1B

hmellor · 1.2B · runs from 2.8 GB

Ilama 3.2 1B is a 1.2B-parameter open language model from hmellor. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

HF Moshiko

kmhf · 7.8B · runs from 16.9 GB

HF Moshiko is a 7.8B-parameter open language model from kmhf. It supports a context window of up to 3,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Deepseek V4 Flash 0731 Spark

0xSero · 60.3B · runs from 26.0 GB

Deepseek V4 Flash 0731 Spark is a 60.3B-parameter open language model from 0xSero in the DeepSeek V4 family. It supports a context window of up to 1,048,576 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Xglm 564M

Meta · 564M · runs from 0.3 GB

Xglm 564M is a 564M-parameter open language model from Meta in the GLM family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MiMo 7B RL

Xiaomi · 7.8B · runs from 3.9 GB

MiMo 7B RL is a 7.8B-parameter open language model from Xiaomi. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

NVIDIA Nemotron Parse 2.0

NVIDIA · 903M · runs from 2.0 GB

NVIDIA Nemotron Parse 2.0 is a sub-1-billion-parameter vision-encoder-decoder model purpose-built for document parsing rather than open-ended chat: given a page image, it outputs structured text with layout classes, bounding boxes, and reading order for elements like titles, paragraphs, tables, charts, and footnotes. It pairs a ViT-H vision encoder based on NVIDIA's C-RADIO with a 10-block mBART decoder, and over its predecessor v1.2 it adds roughly 20,000 new vocabulary tokens for more efficient multilingual (especially CJK and Indic-script) OCR, a dedicated chart class for chart-to-table parsing, and stronger table detection and text extraction. At under a billion parameters it runs on a single modest GPU. License is the OpenMDW License Agreement version 1.1, permitting both commercial and non-commercial use; the bundled tokenizer is separately licensed under CC-BY-4.0. It was published in August 2026.

Gemma 3 270M

Google · 268M · runs from 0.1 GB

Google Gemma 3 270M is a 270-million parameter base (pretrained) model from Google's Gemma 3 family. It is an experimental release intended for research, fine-tuning, and exploring the capabilities of ultra-small language models. The model runs on virtually any hardware with negligible resource requirements. Released under the Gemma license.

GPT Neox Japanese 2.7B

abeja · 2.7B · runs from 5.9 GB

GPT Neox Japanese 2.7B is a 2.7B-parameter open language model from abeja. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Open Llama 7B

openlm-research · 7B · runs from 3.3 GB

Open Llama 7B is a 7B-parameter open language model from openlm-research in the Llama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Deepseek Vl2 Tiny

DeepSeek · 3.4B · runs from 7.4 GB

DeepSeek-VL2-Tiny is the smallest in DeepSeek's VL2 series of Mixture-of-Experts vision-language models, with about 3.4 billion total parameters but only roughly 1.2 billion activated per token. It handles vision-language tasks — visual question answering, OCR, document/table/chart understanding, and visual grounding — below the larger VL2-Small and full VL2 variants. Only a fraction of parameters activate per token, so decoding stays fast even though every expert must still be loaded into memory. It is compact enough to run on a single consumer GPU once quantized. Context length is limited to 4,096 tokens. It is released under DeepSeek's own model license, a custom permissive license with an acceptable-use policy rather than a fully open license like MIT or Apache 2.0. Published in December 2024, it pairs a small DeepSeekMoE-3B backbone with a dynamic image-tiling vision encoder.

Glm 4 9B Chat

Z.ai · 9.4B · runs from 4.4 GB

Glm 4 9B Chat is a 9.4B-parameter open language model from Z.ai in the GLM 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Opt 1.3B

Meta · 1.3B · runs from 2.9 GB

Opt 1.3B is a 1.3B-parameter open language model from Meta. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Stories15M MOE

ggml-org · 36M · runs from 0.3 GB

Stories15M MOE is a 36M-parameter open language model from ggml-org. It supports a context window of up to 256 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Inkling Small DSpark Preview

RadixArk · 1.4B · runs from 0.9 GB

Inkling Small DSpark Preview is a 1.4B-parameter open language model from RadixArk in the Inkling family. It supports a context window of up to 1,048,576 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.