All LLM Models
Browse 1138 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
L3 8B Stheno V3.2
Sao10K · 8.0B · runs from 4.0 GB
10.6K
449L3 8B Stheno V3.2 is a 8.0B-parameter open language model from Sao10K. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 1.5B
Alibaba · 1.5B · runs from 1 GB
Qwen2.5-1.5B is Alibaba's 1.5-billion-parameter entry in the Qwen2.5 family, released as a base pretrained language model rather than an instruction-tuned chat model. Qwen2.5 improved knowledge, coding, and math capabilities over its predecessor through an expanded pretraining corpus, but this checkpoint is the raw base: the model card explicitly discourages using it directly for conversation and recommends fine-tuning (SFT, RLHF, or continued pretraining) first. At this size, it runs comfortably on almost any modern GPU, or even a CPU, without quantization. The model supports a 131,072 token context window and is released under the Apache 2.0 license, allowing unrestricted commercial and research use. It was published in September 2024 alongside the rest of the Qwen2.5 line, which spans from 0.5B to 72B parameters and adds multilingual support for over 29 languages.
HarmBench Llama 2 13B Cls
cais · 13.0B · runs from 5.6 GB
HarmBench Llama 2 13B Cls is a 13.0B-parameter open language model from cais in the Llama 2 family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Falcon 11B
TII UAE · 11.1B · runs from 5.0 GB
Falcon2-11B is TII's base pretrained causal decoder model, not instruction-tuned, with 11 billion parameters trained on over 5,000 billion tokens of the RefinedWeb dataset plus curated corpora spanning English and nine other European languages. Training proceeded in stages that grew the context window from 2,048 to 8,192 tokens before a final high-quality data stage. TII recommends further fine-tuning before production use. At 11 billion parameters it runs on a single consumer GPU, especially once quantized. Context length is 8,192 tokens. It is released under the TII Falcon License 2.0, a custom Apache 2.0-based license that adds an acceptable-use policy for responsible use of the model. It was published in May 2024, and has since been superseded by the Falcon3 series.
Starcoder2 7B
BigCode · 7.2B · runs from 3.5 GB
Starcoder2 7B is a 7.2B-parameter open language model from BigCode in the StarCoder family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.5 9B Claude 4.6 Opus Reasoning Distilled
Jackrong · 9.7B · runs from 4.7 GB
Qwen3.5 9B Claude 4.6 Opus Reasoning Distilled is a 9.7B-parameter open language model from Jackrong in the Qwen 3.5 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.5 35B A3B Base
Alibaba · 36.0B · runs from 10.3 GB
Qwen3.5-35B-A3B-Base is the mixture-of-experts member of Alibaba's Qwen3.5 base family, pairing a 256-expert MoE layer (8 routed plus 1 shared expert per token) with the same Gated DeltaNet and gated-attention hybrid backbone used across the line. It totals roughly 36 billion parameters but activates only about 3 billion per token (the "A3B" in its name), so decoding stays fast even though the full expert set must stay resident in memory. Like its dense siblings it is pretrain-only, for fine-tuning and research, not direct chat; unlike them, its card lists only a pretraining stage, with no post-training pass. It needs a high-end consumer GPU once quantized. Context length is 262,144 tokens natively, extensible to 1,010,000 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2026, alongside four smaller dense Qwen3.5-Base models from 0.8B to 9B parameters.
SmolLM 360M
Hugging Face · 362M · runs from 0.5 GB
SmolLM-360M is Hugging Face's 360-million-parameter base language model, one of three sizes (135M, 360M, and 1.7B) in the original SmolLM family. It is a pretrained model, not instruction-tuned or chat-formatted, trained on 600 billion tokens from the curated Cosmo-Corpus dataset: Cosmopedia v2 synthetic textbooks and stories, Python-Edu educational code samples, and FineWeb-Edu deduplicated educational web text. At this size it is intended as a lightweight research and fine-tuning base rather than a ready chat assistant, and it runs comfortably on a CPU or any consumer GPU. Context length is 2,048 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in July 2024.
InternVL3 1B
OpenGVLab · 938M · runs from 0.3 GB
InternVL3-1B is OpenGVLab's smallest vision-language model in the InternVL3 series, combining an InternViT-300M-448px-V2.5 vision encoder with a Qwen2.5-0.5B language backbone through an MLP projector. Unlike earlier InternVL generations, it uses Native Multimodal Pre-Training, interleaving image-text and video-text data with text-only corpora in a single pretraining stage rather than adapting a finished text model, and adds Variable Visual Position Encoding for better long-context handling. The series extends its capabilities to tool use, GUI agents, industrial image analysis, and 3D vision perception. At under 1 billion parameters, it is small enough to run on a modest consumer GPU or even a CPU. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in April 2025, as the smallest member of a family that scales up to InternVL3-78B.
Carnice V1 9B Hermes Agent Stage2 Merged
kai-os · 9.0B · runs from 4.4 GB
Carnice V1 9B Hermes Agent Stage2 Merged is a 9.0B-parameter open language model from kai-os in the Hermes family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
EuroLLM 1.7B Instruct
utter-project · 1.7B · runs from 1.2 GB
EuroLLM 1.7B Instruct is a 1.7B-parameter open language model from utter-project. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Dobby Mini Unhinged Llama 3.1 8B
SentientAGI · 8.0B · runs from 2.8 GB
Dobby Mini Unhinged Llama 3.1 8B is a 8.0B-parameter open language model from SentientAGI in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2 7B
Alibaba · 7.6B · runs from 3.6 GB
Qwen2 7B is a 7.6B-parameter open language model from Alibaba in the Qwen 2 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Eurus 2 7B PRIME
PRIME-RL · 7.6B · runs from 3.0 GB
Eurus-2-7B-PRIME is a 7.6-billion-parameter reasoning model from the PRIME-RL project, built on Qwen2.5-Math-7B-Base and trained with PRIME (Process Reinforcement through Implicit Rewards), an open-source online reinforcement-learning method that scores intermediate reasoning steps rather than only final answers. It starts from the Eurus-2-7B-SFT checkpoint and is trained further on the Eurus-2-RL-Data set, focused on math and coding problem-solving. The card reports a 16.7% average improvement over the SFT starting point, with over 20% gains on AMC and AIME competition math, enough to surpass the larger Qwen2.5-Math-7B-Instruct on several reasoning benchmarks. At this size it fits on a single consumer GPU. Context length is 4,096 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2024.
InternVL3 8B
OpenGVLab · 7.9B · runs from 2.4 GB
InternVL3-8B is OpenGVLab's roughly 7.9-billion-parameter vision-language model, pairing an InternViT-300M vision encoder with a Qwen2.5-7B language backbone in a ViT-MLP-LLM architecture. It handles general image and video understanding and document analysis, extending into tool use, GUI agent tasks, and 3D scene perception beyond typical captioning. It is comfortably runnable on a single mainstream-to-high-end consumer GPU once quantized. Its language backbone supports a 32,768 token context window. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2025 as part of a 1B-to-78B InternVL3 family sharing the same vision encoder. Its key change versus InternVL2.5 is Native Multimodal Pre-Training, which trains vision and language jointly from the start instead of adapting a language-only model afterward.
LFM2.5 1.2B JP 202606
Liquid AI · 1.2B · runs from 0.9 GB
LFM2.5 1.2B JP 202606 is a 1.2B-parameter open language model from Liquid AI in the LFM2.5 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Saiga Llama3 8B
IlyaGusev · 8.0B · runs from 4.0 GB
Saiga Llama3 8B is a 8.0B-parameter open language model from IlyaGusev in the Llama 3 family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Yi 9B
01.AI · 8.8B · runs from 4.1 GB
Yi-9B is 01.AI's 8.8-billion-parameter base language model, continuously pretrained from Yi-6B on an additional 0.8 trillion tokens as part of the bilingual English/Chinese Yi series (trained on 3 trillion tokens overall). It is a pretrained model, not instruction-tuned; a separate long-context Yi-9B-200K variant exists for extended-context use. The card reports it as the strongest model in its size class among Mistral-7B, SOLAR-10.7B, Gemma-7B, and DeepSeek-Coder-7B-Base, particularly in code, math, common-sense reasoning, and reading comprehension. It fits on a single consumer GPU. Context length is 4,096 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in March 2024.
Pythia 1B
EleutherAI · 1.1B · runs from 0.5 GB
Pythia 1B is a 1.1B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gpt2 Medium
OpenAI · 380M · runs from 0.2 GB
GPT-2 Medium scales the original GPT-2 architecture to 380 million parameters, offering noticeably improved text generation quality over the base 137M variant while remaining extremely lightweight by current standards. It supports the same autoregressive language modeling tasks as its smaller and larger siblings. Like all GPT-2 variants, it runs comfortably on virtually any modern hardware including CPU-only setups, making it an accessible option for learning, prototyping, and lightweight text generation experiments without needing a dedicated GPU.
Granite 4.0 Tiny Preview
IBM · 6.7B · runs from 2.7 GB
Granite 4.0 Tiny Preview is a 6.7B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Granite Guardian 3.3 8B
IBM · 8.2B · runs from 2.9 GB
Granite Guardian 3.3 8B is a 8.2B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
PaddleOCR VL 1.6
PaddlePaddle · 959M · runs from 0.6 GB
PaddleOCR-VL-1.6 is PaddlePaddle's compact, roughly 0.9-billion-parameter vision-language model for document parsing, built on the ERNIE 4.5 line and specialized for OCR, table, formula, chart, and seal/stamp recognition plus text spotting rather than open-domain chat. It upgrades PaddleOCR-VL-1.5 with a region-aware data optimization framework that targets the earlier model's weak spots and a progressive post-training recipe combining curated data selection with reinforcement learning, while staying architecture-compatible with 1.5 for drop-in migration. The card reports a new state-of-the-art 96.33% on OmniDocBench v1.6. At under a billion parameters it runs on a single modest consumer GPU. Context length is 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2026.
C4ai Command R V01
Cohere · 35.0B · runs from 15.9 GB
C4AI Command R is a 35-billion-parameter instruction-tuned model released by Cohere For AI in March 2024. It was built for conversational use with a particular focus on retrieval-augmented generation and tool use, including grounded answers that cite the source documents they draw on, and it was trained to work across ten major languages. It supports a 128K-token context window and is released under a Creative Commons Attribution-NonCommercial 4.0 license, so commercial use is not permitted. At roughly 35 billion parameters, 4-bit quantization needs around 20GB of memory, within reach of a single 24GB consumer GPU.
WhiteRabbitNeo 13B V1
WhiteRabbitNeo · 13B · runs from 7.5 GB
WhiteRabbitNeo 13B V1 is a 13B-parameter open language model from WhiteRabbitNeo. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Laguna XS.2
poolside · 33.4B · runs from 14.6 GB
Laguna XS.2 is a 33.4B-parameter open language model from poolside. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
CodeLlama 7B HF
Meta · 6.7B · runs from 4.2 GB
CodeLlama 7B HF is a 6.7B-parameter open language model from Meta in the Code Llama family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
SmolLM 135M
Hugging Face · 135M · runs from 0.4 GB
SmolLM 135M is the original first-generation small language model from Hugging Face, designed to push the boundaries of what is achievable at extremely low parameter counts. With just 135 million parameters, it was a pioneering effort in making capable language models accessible on the most resource-constrained hardware. While the SmolLM2 and SmolLM3 families have since surpassed it in quality, the original SmolLM 135M remains a useful reference point for research and a practical option for ultra-lightweight deployment scenarios where every megabyte of memory counts.
Pythia 160M
EleutherAI · 213M · runs from 0.1 GB
Pythia 160M is part of EleutherAI's Pythia training suite, a collection of models trained on the same data in the same order at multiple scales to enable rigorous scientific research into how language models learn. At 160 million parameters, it is the smallest model in the suite and runs on virtually any hardware. This model is primarily valuable for researchers studying scaling laws, training dynamics, and emergent capabilities across model sizes. EleutherAI released full training checkpoints, data, and code, making Pythia 160M one of the most transparent and reproducible models available for academic study.
Granite 3.0 1B A400m Instruct
IBM · 1.3B · runs from 1.0 GB
Granite 3.0 1B A400m Instruct is a 1.3B-parameter open language model from IBM in the Granite family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.