All LLM Models
Browse 982 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
LFM2.5 1.2B JP 202606
Liquid AI · 1.2B · runs from 0.9 GB
2.7K
61LFM2.5 1.2B JP 202606 is a 1.2B-parameter open language model from Liquid AI in the LFM2.5 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Saiga Llama3 8B
IlyaGusev · 8.0B · runs from 4.0 GB
Saiga Llama3 8B is a 8.0B-parameter open language model from IlyaGusev in the Llama 3 family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Yi 9B
01.AI · 8.8B · runs from 4.1 GB
Yi-9B is 01.AI's 8.8-billion-parameter base language model, continuously pretrained from Yi-6B on an additional 0.8 trillion tokens as part of the bilingual English/Chinese Yi series (trained on 3 trillion tokens overall). It is a pretrained model, not instruction-tuned; a separate long-context Yi-9B-200K variant exists for extended-context use. The card reports it as the strongest model in its size class among Mistral-7B, SOLAR-10.7B, Gemma-7B, and DeepSeek-Coder-7B-Base, particularly in code, math, common-sense reasoning, and reading comprehension. It fits on a single consumer GPU. Context length is 4,096 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in March 2024.
Pythia 1B
EleutherAI · 1.1B · runs from 0.5 GB
Pythia 1B is a 1.1B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gpt2 Medium
OpenAI · 380M · runs from 0.2 GB
GPT-2 Medium scales the original GPT-2 architecture to 380 million parameters, offering noticeably improved text generation quality over the base 137M variant while remaining extremely lightweight by current standards. It supports the same autoregressive language modeling tasks as its smaller and larger siblings. Like all GPT-2 variants, it runs comfortably on virtually any modern hardware including CPU-only setups, making it an accessible option for learning, prototyping, and lightweight text generation experiments without needing a dedicated GPU.
Granite 4.0 Tiny Preview
IBM · 6.7B · runs from 2.7 GB
Granite 4.0 Tiny Preview is a 6.7B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Granite Guardian 3.3 8B
IBM · 8.2B · runs from 2.9 GB
Granite Guardian 3.3 8B is a 8.2B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
PaddleOCR VL 1.6
PaddlePaddle · 959M · runs from 0.6 GB
PaddleOCR-VL-1.6 is PaddlePaddle's compact, roughly 0.9-billion-parameter vision-language model for document parsing, built on the ERNIE 4.5 line and specialized for OCR, table, formula, chart, and seal/stamp recognition plus text spotting rather than open-domain chat. It upgrades PaddleOCR-VL-1.5 with a region-aware data optimization framework that targets the earlier model's weak spots and a progressive post-training recipe combining curated data selection with reinforcement learning, while staying architecture-compatible with 1.5 for drop-in migration. The card reports a new state-of-the-art 96.33% on OmniDocBench v1.6. At under a billion parameters it runs on a single modest consumer GPU. Context length is 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2026.
WhiteRabbitNeo 13B V1
WhiteRabbitNeo · 13B · runs from 7.5 GB
WhiteRabbitNeo 13B V1 is a 13B-parameter open language model from WhiteRabbitNeo. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
CodeLlama 7B HF
Meta · 6.7B · runs from 4.2 GB
CodeLlama 7B HF is a 6.7B-parameter open language model from Meta in the Code Llama family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
SmolLM 135M
Hugging Face · 135M · runs from 0.4 GB
SmolLM 135M is the original first-generation small language model from Hugging Face, designed to push the boundaries of what is achievable at extremely low parameter counts. With just 135 million parameters, it was a pioneering effort in making capable language models accessible on the most resource-constrained hardware. While the SmolLM2 and SmolLM3 families have since surpassed it in quality, the original SmolLM 135M remains a useful reference point for research and a practical option for ultra-lightweight deployment scenarios where every megabyte of memory counts.
Pythia 160M
EleutherAI · 213M · runs from 0.1 GB
Pythia 160M is part of EleutherAI's Pythia training suite, a collection of models trained on the same data in the same order at multiple scales to enable rigorous scientific research into how language models learn. At 160 million parameters, it is the smallest model in the suite and runs on virtually any hardware. This model is primarily valuable for researchers studying scaling laws, training dynamics, and emergent capabilities across model sizes. EleutherAI released full training checkpoints, data, and code, making Pythia 160M one of the most transparent and reproducible models available for academic study.
Granite 3.0 1B A400m Instruct
IBM · 1.3B · runs from 1.0 GB
Granite 3.0 1B A400m Instruct is a 1.3B-parameter open language model from IBM in the Granite family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mamba 130M HF
State Spaces · 129M · runs from 0.1 GB
Mamba 130M is a state-space model developed by State Spaces that offers a fundamentally different architecture from the Transformer-based models that dominate the LLM landscape. Using selective state-space layers instead of attention, Mamba achieves linear-time inference scaling with sequence length, making it particularly efficient for processing long inputs. At 130 million parameters this is primarily a research and demonstration model, but it showcases the potential of state-space architectures for local deployment. Users interested in exploring alternatives to Transformer-based language models will find Mamba 130M a lightweight and accessible entry point for experimentation.
Cosmos Reason2 8B
NVIDIA · 8.8B · runs from 4.1 GB
Cosmos Reason2-8B is NVIDIA's 8.8-billion-parameter open reasoning vision-language model for physical AI, built on a Qwen3-VL-8B-Instruct backbone and tuned to reason step by step about video and images the way a human would when planning actions in the real world. Rather than just labeling objects, it applies physics, spatio-temporal understanding, and common sense to tasks like robot planning, autonomous-vehicle video captioning, and video-analytics annotation, producing structured outputs such as 2D/3D point localization, bounding boxes, trajectory coordinates, and on-screen OCR text. It ships alongside a smaller 2B variant for edge deployment, while the 8B model needs a capable single GPU or more, less once quantized. Context length is roughly 256,000 tokens, up sharply from 16,000 tokens in the original Cosmos Reason 1. It is released under the NVIDIA Open Model License, a custom license that permits commercial use and derivative models but requires attribution ("Built on NVIDIA Cosmos") and prohibits removing its safety guardrails. It was published in December 2025.
OLMoE 1B 7B 0125 Instruct
Allen AI · 6.9B · runs from 2.5 GB
OLMoE 1B 7B 0125 Instruct is a 6.9B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
JiRackUltra 7B
CMSManhattan · 7.6B · runs from 2.5 GB
JiRackUltra 7B is a 7.6B-parameter open language model from CMSManhattan. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Olmo 3 1025 7B
Allen AI · 7.3B · runs from 3.4 GB
Olmo 3 1025 7B is a 7.3B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 65,536 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Yi 6B Chat
01.AI · 6.1B · runs from 2.9 GB
Yi-6B-Chat is 01.AI's 6-billion-parameter bilingual (English/Chinese) chat model, instruction-tuned from the Yi-6B base model, part of the first-generation Yi series trained from scratch on a 3-trillion-token multilingual corpus. It uses the same Transformer structure popularized by Llama, though 01.AI states it is an independently trained model rather than a Llama derivative, and it was competitive with much larger contemporaries on benchmarks like the Hugging Face Open LLM Leaderboard and C-Eval at release. At 6B parameters it fits a single consumer GPU in half precision, and a much smaller card once 4-bit or 8-bit quantized. Context length is 4,096 tokens; a separate 200K-context variant of the base model is also available for longer documents. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in November 2023.
Dolphin Mistral 24B Venice Edition
dphn · 24.0B · runs from 10.9 GB
Dolphin Mistral 24B Venice Edition is a 24.0B-parameter open language model from dphn in the Phi family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Vicuna 7B V1.5
LMSYS · 7B · runs from 4.3 GB
Vicuna 7B V1.5 is a 7B-parameter open language model from LMSYS in the Vicuna family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
SmolLM 1.7B
Hugging Face · 1.7B · runs from 1.4 GB
SmolLM 1.7B is the largest model in Hugging Face's first-generation SmolLM family. At 1.7 billion parameters, it delivers solid general-purpose text generation in a compact package that runs easily on entry-level hardware, though it has been superseded by the improved SmolLM2 and SmolLM3 series. This model remains a reasonable choice for applications where proven stability matters more than cutting-edge performance. For most new projects, however, users should consider the SmolLM2 1.7B or SmolLM3 3B models, which offer better quality at comparable or only slightly higher resource requirements.
JiRackUltra 14B
CMSManhattan · 14.8B · runs from 4.8 GB
JiRackUltra 14B is a 14.8B-parameter open language model from CMSManhattan. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
VulnLLM R 7B
UCSB-SURFI · 7.6B · runs from 2.5 GB
VulnLLM R 7B is a security-focused model developed by UCSB-SURFI, built on the Qwen2.5-7B base and fine-tuned specifically for vulnerability analysis and security reasoning. With 7.6 billion parameters, it targets tasks like identifying code vulnerabilities, explaining security flaws, and reasoning about attack vectors. This model fills a niche for security researchers and developers who want a locally-hosted assistant for code auditing and vulnerability assessment without sending sensitive code to external APIs. Its specialized training gives it an edge over general-purpose models on security-related tasks, though it is not a replacement for professional security tools. Runs on consumer GPUs with 8 GB of VRAM at typical quantization levels.
Gemma 3 1B Pt
Google · 1000M · runs from 0.3 GB
Gemma 3 1B Pt is a 1000M-parameter open language model from Google in the Gemma 3 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 4B Base
Alibaba · 4.0B · runs from 1.6 GB
Qwen3-4B-Base is Alibaba's 4-billion-parameter base pretrained model from the Qwen3 generation, the successor to Qwen2.5. Like other -Base checkpoints, it is a raw causal language model meant for fine-tuning or research, not direct conversation. Qwen3 was trained on 36 trillion tokens across 119 languages, tripling Qwen2.5's language coverage, using a three-stage pipeline that builds general knowledge, then reasoning skills in code and STEM, then extends context length. At 4B dense parameters, it runs easily on a single consumer GPU, even without heavy quantization. The model supports a 32,768 token context window. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2025. Qwen3 introduced a mix of dense and mixture-of-experts models alongside refinements like QK layernorm for training stability.
Qwen2.5 0.5B
Alibaba · 494M · runs from 0.5 GB
Qwen2.5 0.5B is the smallest base (pretrained) model in Alibaba Cloud's Qwen 2.5 family, with 494 million parameters. As a base model, it is not instruction-tuned and is intended for fine-tuning, research, and as a foundation for custom applications. It supports a 128K token context window. Its minimal size makes it suitable for experimentation, rapid prototyping, and resource-constrained fine-tuning tasks. The model can run on virtually any hardware. Released under the Apache 2.0 license.
Qwen3 1.7B Base
Alibaba · 1.7B · runs from 1.0 GB
Qwen3 1.7B Base is a 1.7-billion parameter pretrained foundation model from Alibaba Cloud's Qwen 3 family. It is a compact base model designed for fine-tuning, research, and custom applications rather than direct conversational use. Its small size makes it accessible for resource-constrained fine-tuning and rapid experimentation. The model can run on virtually any modern GPU and benefits from the improved pretraining data of the Qwen 3 generation. It is suitable as a lightweight foundation for domain-specific fine-tunes and student models in distillation pipelines. Released under the Apache 2.0 license.
Natural Sql 7B
chatdb · 6.9B · runs from 3.2 GB
Natural Sql 7B is a 6.9B-parameter open language model from chatdb. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
MinerU2.5 Pro 2604 1.2B
opendatalab · 1.2B · runs from 0.6 GB
MinerU2.5-Pro-2604-1.2B is opendatalab's 1.2-billion-parameter document parsing model, built on a Qwen2-VL vision-language architecture and specialized for PDF-to-Markdown conversion rather than general chat. It handles complex layouts, tables, and formulas, and the card reports it beating both specialized OCR models like GLM-OCR and PaddleOCR-VL-1.5 and much larger general vision-language models such as Gemini 3 Pro and Qwen3-VL-235B on the OmniDocBench v1.6 leaderboard. The gains over the earlier MinerU 2.5 baseline come entirely from improved training data rather than a change to the 1.2B-parameter architecture. At this size, it fits comfortably on a single modest consumer GPU or even a CPU. Context length is 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2026.