All LLM Models

Browse 739 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Eurus 2 7B PRIME

PRIME-RL · 7.6B · runs from 3.0 GB

Eurus 2 7B PRIME is a 7.6B-parameter open language model from PRIME-RL. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Saiga Llama3 8B

IlyaGusev · 8.0B · runs from 4.0 GB

Saiga Llama3 8B is a 8.0B-parameter open language model from IlyaGusev in the Llama 3 family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Jan v3 4B Base Instruct

janhq · 4.4B · runs from 2.0 GB

Jan v3 4B Base Instruct is a 4.4B-parameter open language model from janhq. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Sqlcoder 7B 2

defog · 6.7B · runs from 4.2 GB

SQLCoder 7B 2 is a 6.7-billion-parameter model from Defog, purpose-built for converting natural-language questions into SQL queries. Fine-tuned specifically on text-to-SQL tasks, it consistently outperforms much larger general-purpose models when the job is generating accurate, executable SQL against real database schemas. For developers and data analysts who regularly query databases, running SQLCoder locally means fast, private SQL generation without sending proprietary schema details to an external API. It works best when provided with table definitions as context and is particularly strong on PostgreSQL, MySQL, and SQLite dialects.

Llama 2 7B Chat HF

Meta · 6.7B · runs from 3.1 GB

Meta Llama 2 7B Chat is a 7-billion parameter instruction-tuned model from Meta's Llama 2 family, optimized for dialogue use cases. It was fine-tuned using supervised fine-tuning and RLHF on top of the Llama 2 7B base model, with a 4K token context window. This model is suitable for basic conversational AI tasks and runs efficiently on consumer GPUs. While newer Llama generations offer improved performance, Llama 2 7B Chat remains a well-understood and widely-supported option for local inference. Released under the Llama 2 Community License.

Yi 6B Chat

01.AI · 6.1B · runs from 2.9 GB

Yi 6B Chat is a 6.1B-parameter open language model from 01.AI in the Yi family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gpt2 Medium

OpenAI · 380M · runs from 0.2 GB

GPT-2 Medium scales the original GPT-2 architecture to 380 million parameters, offering noticeably improved text generation quality over the base 137M variant while remaining extremely lightweight by current standards. It supports the same autoregressive language modeling tasks as its smaller and larger siblings. Like all GPT-2 variants, it runs comfortably on virtually any modern hardware including CPU-only setups, making it an accessible option for learning, prototyping, and lightweight text generation experiments without needing a dedicated GPU.

OLMoE 1B 7B 0125 Instruct

Allen AI · 6.9B · runs from 2.5 GB

OLMoE 1B 7B 0125 Instruct is a 6.9B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Mamba 130M HF

State Spaces · 129M · runs from 0.1 GB

Mamba 130M is a state-space model developed by State Spaces that offers a fundamentally different architecture from the Transformer-based models that dominate the LLM landscape. Using selective state-space layers instead of attention, Mamba achieves linear-time inference scaling with sequence length, making it particularly efficient for processing long inputs. At 130 million parameters this is primarily a research and demonstration model, but it showcases the potential of state-space architectures for local deployment. Users interested in exploring alternatives to Transformer-based language models will find Mamba 130M a lightweight and accessible entry point for experimentation.

Qwen2.5 1.5B

Alibaba · 1.5B · runs from 1 GB

Qwen2.5 1.5B is a 1.5B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen1.5 0.5B Chat

Alibaba · 620M · runs from 0.8 GB

Qwen1.5 0.5B Chat is an early-generation small language model from Alibaba's Qwen series with just 620 million parameters. As one of the smallest models in the Qwen family, it was designed to demonstrate that useful conversational ability is possible even at sub-billion parameter scales. This model runs easily on virtually any hardware including CPUs, older GPUs, and even mobile devices. While its capabilities are limited compared to larger Qwen models, it remains a useful option for embedded applications, rapid prototyping, or situations where minimal resource consumption is the top priority.

Llama3 OpenBioLLM 8B

aaditya · 8B · runs from 3.9 GB

Llama3 OpenBioLLM 8B is a 8B-parameter open language model from aaditya in the Llama 3 family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Meta Llama 3 8B Instruct

Nous Research · 8B · runs from 3.9 GB

Meta Llama 3 8B Instruct is a 8B-parameter open language model from Nous Research in the Llama 3 family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

OpenELM 1 1B Instruct

Apple · 1.1B · runs from 0.5 GB

OpenELM 1 1B Instruct is a 1.1B-parameter open language model from Apple. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 7B IT

Google · 8.5B · runs from 4.0 GB

Google Gemma 7B IT is a 7-billion parameter instruction-tuned model from the original Gemma generation. It is fine-tuned for conversational use and general instruction following, running efficiently on consumer GPUs with 8GB or more of VRAM. As a first-generation Gemma model, it has been superseded by Gemma 2 and Gemma 3 models in quality and capability, but it remains well-supported by inference frameworks. Released under the Gemma license.

Yi 9B

01.AI · 8.8B · runs from 4.1 GB

Yi 9B is a 8.8B-parameter open language model from 01.AI in the Yi family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Salamandra 7B Instruct

BSC-LT · 7.8B · runs from 3.8 GB

Salamandra 7B Instruct is a 7.8-billion-parameter multilingual model developed by the Barcelona Supercomputing Center (BSC-LT) as part of a European initiative to build high-quality open language models. It has particular strength in Iberian languages including Spanish, Catalan, Portuguese, and Basque, while also supporting English and other major European languages. This model is an excellent choice for users who need strong performance in Spanish or other Iberian languages that are often underserved by mainstream LLMs. Running it locally ensures data privacy for sensitive multilingual workflows, and at 7B parameters it fits comfortably on a single consumer GPU with 8 GB or more of VRAM.

Tiny LLM

arnir0 · 13M · runs from 0.3 GB

Tiny LLM is a 13M-parameter open language model from arnir0. It supports a context window of up to 1,024 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Bloom 560M

BigScience · 559M · runs from 0.3 GB

Bloom 560M is a 559M-parameter open language model from BigScience. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Sarvam 30B

sarvamai · 32.2B · runs from 14 GB

Sarvam 30B is a 32.2B-parameter open language model from sarvamai. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

LocoTrainer 4B

LocoreMind · 4.0B · runs from 2.2 GB

LocoTrainer 4B is a 4.0B-parameter open language model from LocoreMind. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatCodeFunctions

NVIDIA Nemotron 3 Nano 4B BF16

NVIDIA · 4.0B · runs from 2.2 GB

NVIDIA Nemotron 3 Nano 4B BF16 is a 4.0B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Prometheus 7B V2.0

prometheus-eval · 7.2B · runs from 3.6 GB

Prometheus 7B V2.0 is a specialized judge model trained by prometheus-eval to evaluate the quality of outputs from other language models. At 7.2 billion parameters, it is designed to score and critique LLM responses against custom rubrics, making it a valuable tool for automated evaluation pipelines and benchmarking. Unlike general-purpose chat models, Prometheus is purpose-built for assessment tasks. It can provide structured feedback on dimensions like helpfulness, accuracy, and coherence. Useful for researchers, developers building LLM applications, and anyone who needs consistent automated evaluation without relying on paid API calls to frontier models. Runs comfortably on most modern GPUs with 8 GB or more of VRAM.

ReaderLM v2

jinaai · 1.5B · runs from 1.0 GB

ReaderLM v2 is a 1.5B-parameter open language model from jinaai. It supports a context window of up to 512,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 68M

JackFram · 68M · runs from 0.0 GB

Llama 68M is a 68M-parameter open language model from JackFram in the Llama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

C4ai Command R V01

Cohere · 35.0B · runs from 15.9 GB

C4ai Command R V01 is a 35.0B-parameter open language model from Cohere in the Command R family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

StableBeluga2

Stability AI · 70B · runs from 20.2 GB

StableBeluga2 is a 70B-parameter open language model from Stability AI. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Amber

LLM360 · 6.7B · runs from 3.2 GB

Amber is a 6.7 billion parameter model from LLM360, an initiative dedicated to full transparency in large language model training. Every aspect of Amber's creation has been publicly documented and released, including the complete training data, all intermediate checkpoints, training code, and evaluation results. This level of openness makes Amber uniquely valuable for researchers studying training dynamics, data influence, and model behavior at scale. For local deployment, it offers solid general-purpose text generation at a size that fits comfortably on mid-range consumer GPUs, though users primarily seeking chat performance may prefer models specifically tuned for instruction following.

Mellum 4B Base

JetBrains · 4.0B · runs from 2.8 GB

Mellum 4B Base is a 4.0B-parameter open language model from JetBrains in the Mellum family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Baichuan2 13B Chat

baichuan-inc · 13B · runs from 3.9 GB

Baichuan2 13B Chat is a 13B-parameter open language model from baichuan-inc in the Baichuan family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.