All LLM Models
Browse 593 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
ERNIE 4.5 21B A3B PT
Baidu · 21B · runs from 6.2 GB
25.6K
165ERNIE 4.5 21B A3B PT is a 21B-parameter open language model from Baidu in the ERNIE family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
INTELLECT 1 Instruct
PrimeIntellect · 10.2B · runs from 3.7 GB
INTELLECT 1 Instruct is a 10.2B-parameter open language model from PrimeIntellect. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mellum2 12B A2.5B Instruct
JetBrains · 12.1B · runs from 5.5 GB
Mellum2 12B A2.5B Instruct is a 12.1B-parameter open language model from JetBrains in the Mellum family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Apertus 8B Instruct 2509
swiss-ai · 8B · runs from 2.8 GB
Apertus 8B Instruct is an open-source instruction-tuned model from Swiss AI, a collaborative research initiative. Built on an 8 billion parameter base, it emphasizes transparency, open data, and European AI sovereignty. For local users, it delivers solid general-purpose chat and instruction-following in a standard 8B footprint that runs well on consumer GPUs with 8 to 10 GB of VRAM, making it a practical choice for those who value open, community-driven model development.
Starcoder2 7B
BigCode · 7.2B · runs from 3.5 GB
Starcoder2 7B is a 7.2B-parameter open language model from BigCode in the StarCoder family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.5 9B Claude 4.6 Opus Reasoning Distilled
Jackrong · 9.7B · runs from 4.7 GB
Qwen3.5 9B Claude 4.6 Opus Reasoning Distilled is a 9.7B-parameter open language model from Jackrong in the Qwen 3.5 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Starcoder2 3B
BigCode · 3.0B · runs from 1.6 GB
Starcoder2 3B is a 3.0B-parameter open language model from BigCode in the StarCoder family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Carnice V1 9B Hermes Agent Stage2 Merged
kai-os · 9.0B · runs from 4.4 GB
Carnice V1 9B Hermes Agent Stage2 Merged is a 9.0B-parameter open language model from kai-os in the Hermes family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
BioMistral 7B
BioMistral · 7B · runs from 3.5 GB
BioMistral 7B is a 7B-parameter open language model from BioMistral in the Mistral family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
LFM2.5 1.2B JP 202606
LiquidAI · 1.2B · runs from 0.9 GB
LFM2.5 1.2B JP 202606 is a 1.2B-parameter open language model from LiquidAI. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron Mini 4B Instruct
NVIDIA · 4B · runs from 1.8 GB
Nemotron Mini 4B Instruct is a 4B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Granite 3.3 8B Instruct
IBM · 8.2B · runs from 2.9 GB
Granite 3.3 8B Instruct is a 8.2B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama Guard 3 8B
Meta · 8.0B · runs from 2.4 GB
Meta Llama Guard 3 8B is an 8-billion parameter safety classifier model built on the Llama 3.1 architecture. Unlike general-purpose chat models, Llama Guard is specifically designed to classify whether prompts or responses contain unsafe content across categories such as violence, sexual content, criminal planning, and other policy violations. The model is intended to be used as a moderation layer in LLM-based applications, providing input and output safety filtering. It follows a taxonomy-based classification approach and can be customized for different safety policies. Released under the Llama 3.1 Community License.
WhiteRabbitNeo 13B V1
WhiteRabbitNeo · 13B · runs from 7.5 GB
WhiteRabbitNeo 13B V1 is a 13B-parameter open language model from WhiteRabbitNeo. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Meta Llama 3 8B
Meta · 8.0B · runs from 3.8 GB
Meta Llama 3 8B is an 8-billion parameter base (pretrained) language model from Meta's Llama 3 release. As a base model, it is not fine-tuned for chat or instructions and is intended for further fine-tuning, research, or as a foundation for custom applications. It uses grouped-query attention and was trained on over 15 trillion tokens. Llama 3 8B supports an 8K token context window and delivers strong benchmark performance across language understanding, reasoning, and coding tasks for its size. It is released under the Meta Llama 3 Community License and runs efficiently on consumer GPUs with 8GB or more of VRAM.
Ternary Bonsai 1.7B Unpacked
prism-ml · 1.7B · runs from 1.3 GB
Ternary Bonsai 1.7B Unpacked is a 1.7B-parameter open language model from prism-ml. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GPT Neox 20B
EleutherAI · 20.7B · runs from 6.3 GB
GPT Neox 20B is a 20.7B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
VulnLLM R 7B
UCSB-SURFI · 7.6B · runs from 2.5 GB
VulnLLM R 7B is a security-focused model developed by UCSB-SURFI, built on the Qwen2.5-7B base and fine-tuned specifically for vulnerability analysis and security reasoning. With 7.6 billion parameters, it targets tasks like identifying code vulnerabilities, explaining security flaws, and reasoning about attack vectors. This model fills a niche for security researchers and developers who want a locally-hosted assistant for code auditing and vulnerability assessment without sending sensitive code to external APIs. Its specialized training gives it an edge over general-purpose models on security-related tasks, though it is not a replacement for professional security tools. Runs on consumer GPUs with 8 GB of VRAM at typical quantization levels.
Gpt2
OpenAI · 137M · runs from 0.1 GB
GPT-2 is the landmark 2019 language model from OpenAI that helped ignite widespread interest in large-scale text generation. At only 137 million parameters it is tiny by modern standards, but it holds an important place in AI history as the model that was initially deemed too dangerous to release in full. Today GPT-2 runs effortlessly on virtually any hardware, including CPUs, making it ideal for educational purposes, experimentation, and understanding transformer fundamentals. It should not be expected to match the quality of modern instruction-tuned models, but it remains a useful teaching tool and conversation starter.
Gemma 7B
Google · 8.5B · runs from 4.0 GB
Google Gemma 7B is a 7-billion parameter base (pretrained) model from the original Gemma generation, Google's first openly available family of language models. It represents Google's initial entry into the open-weight LLM space. While superseded by Gemma 2 and Gemma 3 in terms of benchmark performance, the original Gemma 7B remains a solid foundation model and a useful reference point in the evolution of Google's open models. Released under the Gemma license.
Qwen3 0.6B Base
Alibaba · 596M · runs from 0.7 GB
Qwen3 0.6B Base is the smallest pretrained foundation model in Alibaba Cloud's Qwen 3 family, with approximately 600 million parameters. As a base model, it is not tuned for chat or instructions and is intended for fine-tuning, research, and experimentation. Its minimal size makes it suitable for rapid prototyping and resource-constrained training experiments. The model runs on virtually any hardware, including CPU-only setups. It is useful for educational purposes, architecture exploration, and as a compact foundation for task-specific fine-tuning where model size is a primary constraint. Released under the Apache 2.0 license.
Kappa 20B 131k
eousphoros · 20.9B · runs from 9.3 GB
Kappa 20B 131k is a 20.9B-parameter open language model from eousphoros. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Falcon Mamba 7B
TII UAE · 7.3B · runs from 2.7 GB
Falcon Mamba 7B is a 7.3B-parameter open language model from TII UAE in the Falcon family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Internlm3 8B Instruct
InternLM · 8.8B · runs from 3.4 GB
Internlm3 8B Instruct is a 8.8B-parameter open language model from InternLM in the InternLM family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Ternary Bonsai 4B Unpacked
prism-ml · 4.0B · runs from 2.2 GB
Ternary Bonsai 4B Unpacked is a 4.0B-parameter open language model from prism-ml. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 1.7B Base
Alibaba · 1.7B · runs from 1.0 GB
Qwen3 1.7B Base is a 1.7-billion parameter pretrained foundation model from Alibaba Cloud's Qwen 3 family. It is a compact base model designed for fine-tuning, research, and custom applications rather than direct conversational use. Its small size makes it accessible for resource-constrained fine-tuning and rapid experimentation. The model can run on virtually any modern GPU and benefits from the improved pretraining data of the Qwen 3 generation. It is suitable as a lightweight foundation for domain-specific fine-tunes and student models in distillation pipelines. Released under the Apache 2.0 license.
NVIDIA Nemotron Nano 9B v2
NVIDIA · 8.9B · runs from 4.5 GB
NVIDIA Nemotron Nano 9B v2 is a compact yet capable chat model from NVIDIA, packing 8.9 billion parameters into a size that runs comfortably on a wide range of consumer GPUs. Built on NVIDIA's Nemotron architecture, it delivers strong instruction-following and conversational performance while keeping VRAM requirements modest. This second-generation Nano model reflects NVIDIA's push to make high-quality language models accessible on local hardware. It's an excellent starting point for users who want a responsive, general-purpose assistant without needing top-tier GPU memory.
Gemma 2B IT
Google · 2.5B · runs from 1.2 GB
Gemma 2B IT is a 2.5B-parameter open language model from Google in the Gemma 2 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Yi 6B
01.AI · 6.1B · runs from 2.9 GB
Yi 6B is a 6.1B-parameter open language model from 01.AI in the Yi family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
TinyLlama 1.1B Intermediate Step 1431k 3T
TinyLlama · 1.1B · runs from 0.8 GB
TinyLlama 1.1B Intermediate Step 1431k 3T is a 1.1B-parameter open language model from TinyLlama in the TinyLlama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.