All LLM Models
Browse 856 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Gemma 7B IT
Google · 8.5B · runs from 4.0 GB
21.4K
1.2KGoogle Gemma 7B IT is a 7-billion parameter instruction-tuned model from the original Gemma generation. It is fine-tuned for conversational use and general instruction following, running efficiently on consumer GPUs with 8GB or more of VRAM. As a first-generation Gemma model, it has been superseded by Gemma 2 and Gemma 3 models in quality and capability, but it remains well-supported by inference frameworks. Released under the Gemma license.
Yi 9B
01.AI · 8.8B · runs from 4.1 GB
Yi 9B is a 8.8B-parameter open language model from 01.AI in the Yi family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Salamandra 7B Instruct
BSC-LT · 7.8B · runs from 3.8 GB
Salamandra 7B Instruct is a 7.8-billion-parameter multilingual model developed by the Barcelona Supercomputing Center (BSC-LT) as part of a European initiative to build high-quality open language models. It has particular strength in Iberian languages including Spanish, Catalan, Portuguese, and Basque, while also supporting English and other major European languages. This model is an excellent choice for users who need strong performance in Spanish or other Iberian languages that are often underserved by mainstream LLMs. Running it locally ensures data privacy for sensitive multilingual workflows, and at 7B parameters it fits comfortably on a single consumer GPU with 8 GB or more of VRAM.
Tiny LLM
arnir0 · 13M · runs from 0.3 GB
Tiny LLM is a 13M-parameter open language model from arnir0. It supports a context window of up to 1,024 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Bloom 560M
BigScience · 559M · runs from 0.3 GB
Bloom 560M is a 559M-parameter open language model from BigScience. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Sarvam 30B
sarvamai · 32.2B · runs from 14 GB
Sarvam 30B is a 32.2B-parameter open language model from sarvamai. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
LocoTrainer 4B
LocoreMind · 4.0B · runs from 2.2 GB
LocoTrainer 4B is a 4.0B-parameter open language model from LocoreMind. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron 3 Nano 4B BF16
NVIDIA · 4.0B · runs from 2.2 GB
NVIDIA Nemotron 3 Nano 4B BF16 is a 4.0B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Prometheus 7B V2.0
prometheus-eval · 7.2B · runs from 3.6 GB
Prometheus 7B V2.0 is a specialized judge model trained by prometheus-eval to evaluate the quality of outputs from other language models. At 7.2 billion parameters, it is designed to score and critique LLM responses against custom rubrics, making it a valuable tool for automated evaluation pipelines and benchmarking. Unlike general-purpose chat models, Prometheus is purpose-built for assessment tasks. It can provide structured feedback on dimensions like helpfulness, accuracy, and coherence. Useful for researchers, developers building LLM applications, and anyone who needs consistent automated evaluation without relying on paid API calls to frontier models. Runs comfortably on most modern GPUs with 8 GB or more of VRAM.
ReaderLM v2
jinaai · 1.5B · runs from 1.0 GB
ReaderLM v2 is a 1.5B-parameter open language model from jinaai. It supports a context window of up to 512,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 68M
JackFram · 68M · runs from 0.0 GB
Llama 68M is a 68M-parameter open language model from JackFram in the Llama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
C4ai Command R V01
Cohere · 35.0B · runs from 15.9 GB
C4ai Command R V01 is a 35.0B-parameter open language model from Cohere in the Command R family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
StableBeluga2
Stability AI · 70B · runs from 20.2 GB
StableBeluga2 is a 70B-parameter open language model from Stability AI. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Amber
LLM360 · 6.7B · runs from 3.2 GB
Amber is a 6.7 billion parameter model from LLM360, an initiative dedicated to full transparency in large language model training. Every aspect of Amber's creation has been publicly documented and released, including the complete training data, all intermediate checkpoints, training code, and evaluation results. This level of openness makes Amber uniquely valuable for researchers studying training dynamics, data influence, and model behavior at scale. For local deployment, it offers solid general-purpose text generation at a size that fits comfortably on mid-range consumer GPUs, though users primarily seeking chat performance may prefer models specifically tuned for instruction following.
Mellum 4B Base
JetBrains · 4.0B · runs from 2.8 GB
Mellum 4B Base is a 4.0B-parameter open language model from JetBrains in the Mellum family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Baichuan2 13B Chat
baichuan-inc · 13B · runs from 3.9 GB
Baichuan2 13B Chat is a 13B-parameter open language model from baichuan-inc in the Baichuan family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
SmolLM3 3B Base
Hugging Face · 3B · runs from 1.3 GB
SmolLM3 3B Base is the pretrained foundation model from Hugging Face's third-generation SmolLM family. Without instruction tuning or chat alignment, it serves as a versatile starting point for researchers and developers who want to fine-tune the model for specific domains, tasks, or behavioral profiles. With 3 billion parameters and the architectural improvements introduced in SmolLM3, this base model offers strong general language capabilities in a package that remains practical to train and adapt on consumer-grade hardware. It is an excellent choice for custom fine-tuning projects where off-the-shelf chat behavior is not needed.
Gpt2 Large
OpenAI · 812M · runs from 0.4 GB
Gpt2 Large is a 812M-parameter open language model from OpenAI. It supports a context window of up to 1,024 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Pythia 410M
EleutherAI · 506M · runs from 0.2 GB
Pythia 410M is a 506M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Dolphin 2.9.1 Yi 1.5 34B
dphn · 34.4B · runs from 10.3 GB
Dolphin 2.9.1 Yi 1.5 34B is a 34.4-billion parameter chat model created by Eric Hartford's Dolphin project, fine-tuned from 01.AI's Yi 1.5 34B base. The Dolphin series is known for producing uncensored fine-tunes that remove alignment-based refusals, giving users more direct and unrestricted model responses. This model combines the strong bilingual capabilities of Yi 1.5 with Dolphin's open fine-tuning approach. It requires a GPU with at least 24GB of VRAM for quantized local inference and is popular among users who prefer models without built-in content restrictions.
Falcon 11B
TII UAE · 11.1B · runs from 5.0 GB
Falcon 11B is a 11.1B-parameter open language model from TII UAE in the Falcon family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 0.5B
Alibaba · 494M · runs from 0.5 GB
Qwen2.5 0.5B is the smallest base (pretrained) model in Alibaba Cloud's Qwen 2.5 family, with 494 million parameters. As a base model, it is not instruction-tuned and is intended for fine-tuning, research, and as a foundation for custom applications. It supports a 128K token context window. Its minimal size makes it suitable for experimentation, rapid prototyping, and resource-constrained fine-tuning tasks. The model can run on virtually any hardware. Released under the Apache 2.0 license.
Qwen3 8B Base
Alibaba · 8.2B · runs from 4.1 GB
Qwen3 8B Base is an 8.2-billion parameter pretrained foundation model from Alibaba Cloud's Qwen 3 series. As a base model, it is not instruction-tuned and is intended for fine-tuning, research, and as a starting point for custom downstream applications. It was trained on a large multilingual corpus with improved data quality and training methodology compared to the Qwen 2.5 generation. The model runs efficiently on consumer GPUs with 8GB or more of VRAM and serves as the foundation for the Qwen3 8B instruction-tuned variant and community fine-tunes. It is a strong choice for practitioners building specialized models through further training. Released under the Apache 2.0 license.
Qwen3Guard Gen 8B
Alibaba · 8.2B · runs from 4.1 GB
Qwen3Guard Gen 8B is a 8.2B-parameter open language model from Alibaba in the Qwen 3 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
C4ai Command R V01
CohereForAI · 35.0B · runs from 16.4 GB
C4ai Command R V01 is a 35.0B-parameter open language model from CohereForAI in the Command R family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen1.5 14B Chat
Alibaba · 14.2B · runs from 8 GB
Qwen1.5 14B Chat is a 14.2B-parameter open language model from Alibaba in the Qwen family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gpt2 Xl
OpenAI · 1.6B · runs from 0.7 GB
GPT-2 XL is the largest variant of the GPT-2 family at 1.6 billion parameters, representing the full release of the model OpenAI originally withheld over safety concerns in 2019. It produces the most coherent and capable outputs of the GPT-2 lineup, though it remains far behind modern multi-billion-parameter instruction-tuned models. At its size, GPT-2 XL still runs easily on most consumer GPUs and even on CPUs with reasonable speed, making it useful for experimentation, fine-tuning projects, and as a baseline for comparing against newer architectures. It requires roughly 3 GB of VRAM at full precision.
SmolLM2 1.7B
Hugging Face · 1.7B · runs from 1.4 GB
SmolLM2 1.7B is the base pretrained model from Hugging Face's second-generation SmolLM family. Unlike the instruct variant, this model has not been fine-tuned for chat or instruction following, making it a strong foundation for custom fine-tuning, domain adaptation, or research into small-scale language model behavior. At 1.7 billion parameters, it provides meaningful language understanding and generation capabilities while remaining lightweight enough to train and experiment with on consumer hardware. Researchers and developers who want full control over downstream behavior will find this base model more flexible than the instruction-tuned version.
SmolLM2 135M
Hugging Face · 135M · runs from 0.4 GB
SmolLM2 135M is one of the smallest capable language models available, developed by Hugging Face as part of their SmolLM2 family. With just 135 million parameters, it requires virtually no VRAM and can run on almost any hardware, making it an excellent starting point for researchers experimenting with language model behavior, fine-tuning workflows, or edge deployment scenarios. Despite its tiny footprint, SmolLM2 135M benefits from improved training data and techniques compared to its first-generation predecessor. It is best suited for lightweight text generation tasks, prototyping, and educational purposes rather than production-grade applications.
Qwen2 72B Instruct
Alibaba · 72.7B · runs from 21.0 GB
Qwen2 72B Instruct is a 72.7B-parameter open language model from Alibaba in the Qwen 2 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.