All LLM Models

Browse 1242 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Pythia 160M

EleutherAI · 213M · runs from 0.1 GB

Pythia 160M is part of EleutherAI's Pythia training suite, a collection of models trained on the same data in the same order at multiple scales to enable rigorous scientific research into how language models learn. At 160 million parameters, it is the smallest model in the suite and runs on virtually any hardware. This model is primarily valuable for researchers studying scaling laws, training dynamics, and emergent capabilities across model sizes. EleutherAI released full training checkpoints, data, and code, making Pythia 160M one of the most transparent and reproducible models available for academic study.

Granite 3.0 1B A400m Instruct

IBM · 1.3B · runs from 1.0 GB

Granite 3.0 1B A400m Instruct is a 1.3B-parameter open language model from IBM in the Granite family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Mamba 130M HF

State Spaces · 129M · runs from 0.1 GB

Mamba 130M is a state-space model developed by State Spaces that offers a fundamentally different architecture from the Transformer-based models that dominate the LLM landscape. Using selective state-space layers instead of attention, Mamba achieves linear-time inference scaling with sequence length, making it particularly efficient for processing long inputs. At 130 million parameters this is primarily a research and demonstration model, but it showcases the potential of state-space architectures for local deployment. Users interested in exploring alternatives to Transformer-based language models will find Mamba 130M a lightweight and accessible entry point for experimentation.

Cosmos Reason2 8B

NVIDIA · 8.8B · runs from 4.1 GB

Cosmos Reason2-8B is NVIDIA's 8.8-billion-parameter open reasoning vision-language model for physical AI, built on a Qwen3-VL-8B-Instruct backbone and tuned to reason step by step about video and images the way a human would when planning actions in the real world. Rather than just labeling objects, it applies physics, spatio-temporal understanding, and common sense to tasks like robot planning, autonomous-vehicle video captioning, and video-analytics annotation, producing structured outputs such as 2D/3D point localization, bounding boxes, trajectory coordinates, and on-screen OCR text. It ships alongside a smaller 2B variant for edge deployment, while the 8B model needs a capable single GPU or more, less once quantized. Context length is roughly 256,000 tokens, up sharply from 16,000 tokens in the original Cosmos Reason 1. It is released under the NVIDIA Open Model License, a custom license that permits commercial use and derivative models but requires attribution ("Built on NVIDIA Cosmos") and prohibits removing its safety guardrails. It was published in December 2025.

VisionChatReasoning

OLMoE 1B 7B 0125 Instruct

Allen AI · 6.9B · runs from 2.5 GB

OLMoE 1B 7B 0125 Instruct is a 6.9B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

JiRackUltra 7B

CMSManhattan · 7.6B · runs from 2.5 GB

JiRackUltra 7B is a 7.6B-parameter open language model from CMSManhattan. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Olmo 3 1025 7B

Allen AI · 7.3B · runs from 3.4 GB

Olmo 3 1025 7B is a 7.3B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 65,536 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Yi 6B Chat

01.AI · 6.1B · runs from 2.9 GB

Yi-6B-Chat is 01.AI's 6-billion-parameter bilingual (English/Chinese) chat model, instruction-tuned from the Yi-6B base model, part of the first-generation Yi series trained from scratch on a 3-trillion-token multilingual corpus. It uses the same Transformer structure popularized by Llama, though 01.AI states it is an independently trained model rather than a Llama derivative, and it was competitive with much larger contemporaries on benchmarks like the Hugging Face Open LLM Leaderboard and C-Eval at release. At 6B parameters it fits a single consumer GPU in half precision, and a much smaller card once 4-bit or 8-bit quantized. Context length is 4,096 tokens; a separate 200K-context variant of the base model is also available for longer documents. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in November 2023.

Dolphin Mistral 24B Venice Edition

dphn · 24.0B · runs from 10.9 GB

Dolphin Mistral 24B Venice Edition is a 24.0B-parameter open language model from dphn in the Phi family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Vicuna 7B V1.5

LMSYS · 7B · runs from 4.3 GB

Vicuna 7B V1.5 is a 7B-parameter open language model from LMSYS in the Vicuna family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

SmolLM 1.7B

Hugging Face · 1.7B · runs from 1.4 GB

SmolLM 1.7B is the largest model in Hugging Face's first-generation SmolLM family. At 1.7 billion parameters, it delivers solid general-purpose text generation in a compact package that runs easily on entry-level hardware, though it has been superseded by the improved SmolLM2 and SmolLM3 series. This model remains a reasonable choice for applications where proven stability matters more than cutting-edge performance. For most new projects, however, users should consider the SmolLM2 1.7B or SmolLM3 3B models, which offer better quality at comparable or only slightly higher resource requirements.

JiRackUltra 14B

CMSManhattan · 14.8B · runs from 4.8 GB

JiRackUltra 14B is a 14.8B-parameter open language model from CMSManhattan. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

VulnLLM R 7B

UCSB-SURFI · 7.6B · runs from 2.5 GB

VulnLLM R 7B is a security-focused model developed by UCSB-SURFI, built on the Qwen2.5-7B base and fine-tuned specifically for vulnerability analysis and security reasoning. With 7.6 billion parameters, it targets tasks like identifying code vulnerabilities, explaining security flaws, and reasoning about attack vectors. This model fills a niche for security researchers and developers who want a locally-hosted assistant for code auditing and vulnerability assessment without sending sensitive code to external APIs. Its specialized training gives it an edge over general-purpose models on security-related tasks, though it is not a replacement for professional security tools. Runs on consumer GPUs with 8 GB of VRAM at typical quantization levels.

Gemma 3 1B Pt

Google · 1000M · runs from 0.3 GB

Gemma 3 1B Pt is a 1000M-parameter open language model from Google in the Gemma 3 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3 4B Base

Alibaba · 4.0B · runs from 1.6 GB

Qwen3-4B-Base is Alibaba's 4-billion-parameter base pretrained model from the Qwen3 generation, the successor to Qwen2.5. Like other -Base checkpoints, it is a raw causal language model meant for fine-tuning or research, not direct conversation. Qwen3 was trained on 36 trillion tokens across 119 languages, tripling Qwen2.5's language coverage, using a three-stage pipeline that builds general knowledge, then reasoning skills in code and STEM, then extends context length. At 4B dense parameters, it runs easily on a single consumer GPU, even without heavy quantization. The model supports a 32,768 token context window. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2025. Qwen3 introduced a mix of dense and mixture-of-experts models alongside refinements like QK layernorm for training stability.

Qwen2.5 0.5B

Alibaba · 494M · runs from 0.5 GB

Qwen2.5 0.5B is the smallest base (pretrained) model in Alibaba Cloud's Qwen 2.5 family, with 494 million parameters. As a base model, it is not instruction-tuned and is intended for fine-tuning, research, and as a foundation for custom applications. It supports a 128K token context window. Its minimal size makes it suitable for experimentation, rapid prototyping, and resource-constrained fine-tuning tasks. The model can run on virtually any hardware. Released under the Apache 2.0 license.

Qwen3 1.7B Base

Alibaba · 1.7B · runs from 1.0 GB

Qwen3 1.7B Base is a 1.7-billion parameter pretrained foundation model from Alibaba Cloud's Qwen 3 family. It is a compact base model designed for fine-tuning, research, and custom applications rather than direct conversational use. Its small size makes it accessible for resource-constrained fine-tuning and rapid experimentation. The model can run on virtually any modern GPU and benefits from the improved pretraining data of the Qwen 3 generation. It is suitable as a lightweight foundation for domain-specific fine-tunes and student models in distillation pipelines. Released under the Apache 2.0 license.

Natural Sql 7B

chatdb · 6.9B · runs from 3.2 GB

Natural Sql 7B is a 6.9B-parameter open language model from chatdb. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MinerU2.5 Pro 2604 1.2B

opendatalab · 1.2B · runs from 0.6 GB

MinerU2.5-Pro-2604-1.2B is opendatalab's 1.2-billion-parameter document parsing model, built on a Qwen2-VL vision-language architecture and specialized for PDF-to-Markdown conversion rather than general chat. It handles complex layouts, tables, and formulas, and the card reports it beating both specialized OCR models like GLM-OCR and PaddleOCR-VL-1.5 and much larger general vision-language models such as Gemini 3 Pro and Qwen3-VL-235B on the OmniDocBench v1.6 leaderboard. The gains over the earlier MinerU 2.5 baseline come entirely from improved training data rather than a change to the 1.2B-parameter architecture. At this size, it fits comfortably on a single modest consumer GPU or even a CPU. Context length is 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2026.

Meta Llama 3 8B

Meta · 8.0B · runs from 3.8 GB

Meta Llama 3 8B is an 8-billion parameter base (pretrained) language model from Meta's Llama 3 release. As a base model, it is not fine-tuned for chat or instructions and is intended for further fine-tuning, research, or as a foundation for custom applications. It uses grouped-query attention and was trained on over 15 trillion tokens. Llama 3 8B supports an 8K token context window and delivers strong benchmark performance across language understanding, reasoning, and coding tasks for its size. It is released under the Meta Llama 3 Community License and runs efficiently on consumer GPUs with 8GB or more of VRAM.

Kappa 20B 131k

eousphoros · 20.9B · runs from 9.3 GB

Kappa 20B 131k is a 20.9B-parameter open language model from eousphoros. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama Guard 3 1B

alpindale · 1.5B · runs from 0.8 GB

Llama Guard 3 1B is a 1.5B-parameter open language model from alpindale in the Llama family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Falcon Mamba 7B

TII UAE · 7.3B · runs from 2.7 GB

Falcon Mamba 7B is a 7.3B-parameter open language model from TII UAE in the Falcon family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.5 2B Base

Alibaba · 2.3B · runs from 1.4 GB

Qwen3.5-2B-Base is a 2.3-billion-parameter dense checkpoint in Alibaba's Qwen3.5 base model family, a native vision-language foundation model rather than a text-only model with vision bolted on. It is pretrain-only: fine-tuning, in-context-learning, or further research, not direct conversation, though its control tokens are compatible with the official chat template for efficient LoRA-style adaptation. Its hybrid architecture pairs Gated DeltaNet linear attention with periodic gated full-attention layers. At 2.3B parameters, it runs comfortably on a single consumer GPU, even unquantized. The model supports a native 262,144 token context window, extensible up to 1,010,000 tokens. It is released under the Apache 2.0 license, and was published in February 2026. Qwen3.5 introduced early-fusion multimodal pretraining that Alibaba says outperforms the separate Qwen3-VL models on reasoning, coding, and visual understanding.

SmolLM3 3B Base

Hugging Face · 3.1B · runs from 1.3 GB

SmolLM3 3B Base is the pretrained foundation model from Hugging Face's third-generation SmolLM family. Without instruction tuning or chat alignment, it serves as a versatile starting point for researchers and developers who want to fine-tune the model for specific domains, tasks, or behavioral profiles. With 3 billion parameters and the architectural improvements introduced in SmolLM3, this base model offers strong general language capabilities in a package that remains practical to train and adapt on consumer-grade hardware. It is an excellent choice for custom fine-tuning projects where off-the-shelf chat behavior is not needed.

Qwen3 30B A3B Base

Alibaba · 30.5B · runs from 12.2 GB

Qwen3 30B A3B Base is a 30.5B-parameter open language model from Alibaba in the Qwen 3 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 3.1 8B

Meta · 8.0B · runs from 3.6 GB

Meta Llama 3.1 8B is an 8-billion parameter base (pretrained) model from the Llama 3.1 family. It is not instruction-tuned and is intended for fine-tuning, research, and custom downstream applications. Compared to Llama 3 8B, it extends the context window to 128K tokens and benefits from improved training data and methodology. The model uses grouped-query attention and was trained on a multilingual corpus. It is released under the Llama 3.1 Community License and is widely used as a foundation for community fine-tunes and specialized models.

Qwen3 0.6B Heretic Abliterated Uncensored

DavidAU · 596M · runs from 0.7 GB

Qwen3 0.6B Heretic Abliterated Uncensored is a 596M-parameter open language model from DavidAU in the Qwen 3 family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qianfan OCR

Baidu · 4.7B · runs from 2.5 GB

Qianfan-OCR is Baidu's 4.7-billion-parameter vision-language model for document intelligence rather than general chat. It pairs a Qianfan-ViT vision encoder with a Qwen3-4B language backbone, doing direct image-to-Markdown conversion alongside table extraction, chart understanding, and document Q&A in one end-to-end model instead of a multi-stage pipeline. It also has an optional "Layout-as-Thought" mode that reasons about page layout before producing output. At under 5 billion parameters, it runs on a single consumer or prosumer GPU once quantized. Context length is 32,768 tokens, extendable further per Baidu's documentation. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in March 2026, Baidu reports it as its top-scoring end-to-end model on public document-parsing benchmarks, supporting 192 languages.

OLMo 2 0325 32B Instruct

Allen AI · 32.2B · runs from 14.5 GB

OLMo 2 0325 32B Instruct is a 32.2B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.