All LLM Models

Browse 1214 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Dolphin Mistral 24B Venice Edition

dphn · 24.0B · runs from 10.9 GB

Dolphin Mistral 24B Venice Edition is a 24.0B-parameter open language model from dphn in the Phi family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Vicuna 7B V1.5

LMSYS · 7B · runs from 4.3 GB

Vicuna 7B V1.5 is a 7B-parameter open language model from LMSYS in the Vicuna family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

SmolLM 1.7B

Hugging Face · 1.7B · runs from 1.4 GB

SmolLM 1.7B is the largest model in Hugging Face's first-generation SmolLM family. At 1.7 billion parameters, it delivers solid general-purpose text generation in a compact package that runs easily on entry-level hardware, though it has been superseded by the improved SmolLM2 and SmolLM3 series. This model remains a reasonable choice for applications where proven stability matters more than cutting-edge performance. For most new projects, however, users should consider the SmolLM2 1.7B or SmolLM3 3B models, which offer better quality at comparable or only slightly higher resource requirements.

JiRackUltra 14B

CMSManhattan · 14.8B · runs from 4.8 GB

JiRackUltra 14B is a 14.8B-parameter open language model from CMSManhattan. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

VulnLLM R 7B

UCSB-SURFI · 7.6B · runs from 2.5 GB

VulnLLM R 7B is a security-focused model developed by UCSB-SURFI, built on the Qwen2.5-7B base and fine-tuned specifically for vulnerability analysis and security reasoning. With 7.6 billion parameters, it targets tasks like identifying code vulnerabilities, explaining security flaws, and reasoning about attack vectors. This model fills a niche for security researchers and developers who want a locally-hosted assistant for code auditing and vulnerability assessment without sending sensitive code to external APIs. Its specialized training gives it an edge over general-purpose models on security-related tasks, though it is not a replacement for professional security tools. Runs on consumer GPUs with 8 GB of VRAM at typical quantization levels.

Gemma 3 1B Pt

Google · 1000M · runs from 0.3 GB

Gemma 3 1B Pt is a 1000M-parameter open language model from Google in the Gemma 3 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3 4B Base

Alibaba · 4.0B · runs from 1.6 GB

Qwen3-4B-Base is Alibaba's 4-billion-parameter base pretrained model from the Qwen3 generation, the successor to Qwen2.5. Like other -Base checkpoints, it is a raw causal language model meant for fine-tuning or research, not direct conversation. Qwen3 was trained on 36 trillion tokens across 119 languages, tripling Qwen2.5's language coverage, using a three-stage pipeline that builds general knowledge, then reasoning skills in code and STEM, then extends context length. At 4B dense parameters, it runs easily on a single consumer GPU, even without heavy quantization. The model supports a 32,768 token context window. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2025. Qwen3 introduced a mix of dense and mixture-of-experts models alongside refinements like QK layernorm for training stability.

Qwen2.5 0.5B

Alibaba · 494M · runs from 0.5 GB

Qwen2.5 0.5B is the smallest base (pretrained) model in Alibaba Cloud's Qwen 2.5 family, with 494 million parameters. As a base model, it is not instruction-tuned and is intended for fine-tuning, research, and as a foundation for custom applications. It supports a 128K token context window. Its minimal size makes it suitable for experimentation, rapid prototyping, and resource-constrained fine-tuning tasks. The model can run on virtually any hardware. Released under the Apache 2.0 license.

Qwen3 1.7B Base

Alibaba · 1.7B · runs from 1.0 GB

Qwen3 1.7B Base is a 1.7-billion parameter pretrained foundation model from Alibaba Cloud's Qwen 3 family. It is a compact base model designed for fine-tuning, research, and custom applications rather than direct conversational use. Its small size makes it accessible for resource-constrained fine-tuning and rapid experimentation. The model can run on virtually any modern GPU and benefits from the improved pretraining data of the Qwen 3 generation. It is suitable as a lightweight foundation for domain-specific fine-tunes and student models in distillation pipelines. Released under the Apache 2.0 license.

Natural Sql 7B

chatdb · 6.9B · runs from 3.2 GB

Natural Sql 7B is a 6.9B-parameter open language model from chatdb. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MinerU2.5 Pro 2604 1.2B

opendatalab · 1.2B · runs from 0.6 GB

MinerU2.5-Pro-2604-1.2B is opendatalab's 1.2-billion-parameter document parsing model, built on a Qwen2-VL vision-language architecture and specialized for PDF-to-Markdown conversion rather than general chat. It handles complex layouts, tables, and formulas, and the card reports it beating both specialized OCR models like GLM-OCR and PaddleOCR-VL-1.5 and much larger general vision-language models such as Gemini 3 Pro and Qwen3-VL-235B on the OmniDocBench v1.6 leaderboard. The gains over the earlier MinerU 2.5 baseline come entirely from improved training data rather than a change to the 1.2B-parameter architecture. At this size, it fits comfortably on a single modest consumer GPU or even a CPU. Context length is 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2026.

Meta Llama 3 8B

Meta · 8.0B · runs from 3.8 GB

Meta Llama 3 8B is an 8-billion parameter base (pretrained) language model from Meta's Llama 3 release. As a base model, it is not fine-tuned for chat or instructions and is intended for further fine-tuning, research, or as a foundation for custom applications. It uses grouped-query attention and was trained on over 15 trillion tokens. Llama 3 8B supports an 8K token context window and delivers strong benchmark performance across language understanding, reasoning, and coding tasks for its size. It is released under the Meta Llama 3 Community License and runs efficiently on consumer GPUs with 8GB or more of VRAM.

Kappa 20B 131k

eousphoros · 20.9B · runs from 9.3 GB

Kappa 20B 131k is a 20.9B-parameter open language model from eousphoros. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama Guard 3 1B

alpindale · 1.5B · runs from 0.8 GB

Llama Guard 3 1B is a 1.5B-parameter open language model from alpindale in the Llama family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Falcon Mamba 7B

TII UAE · 7.3B · runs from 2.7 GB

Falcon Mamba 7B is a 7.3B-parameter open language model from TII UAE in the Falcon family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.5 2B Base

Alibaba · 2.3B · runs from 1.4 GB

Qwen3.5-2B-Base is a 2.3-billion-parameter dense checkpoint in Alibaba's Qwen3.5 base model family, a native vision-language foundation model rather than a text-only model with vision bolted on. It is pretrain-only: fine-tuning, in-context-learning, or further research, not direct conversation, though its control tokens are compatible with the official chat template for efficient LoRA-style adaptation. Its hybrid architecture pairs Gated DeltaNet linear attention with periodic gated full-attention layers. At 2.3B parameters, it runs comfortably on a single consumer GPU, even unquantized. The model supports a native 262,144 token context window, extensible up to 1,010,000 tokens. It is released under the Apache 2.0 license, and was published in February 2026. Qwen3.5 introduced early-fusion multimodal pretraining that Alibaba says outperforms the separate Qwen3-VL models on reasoning, coding, and visual understanding.

SmolLM3 3B Base

Hugging Face · 3.1B · runs from 1.3 GB

SmolLM3 3B Base is the pretrained foundation model from Hugging Face's third-generation SmolLM family. Without instruction tuning or chat alignment, it serves as a versatile starting point for researchers and developers who want to fine-tune the model for specific domains, tasks, or behavioral profiles. With 3 billion parameters and the architectural improvements introduced in SmolLM3, this base model offers strong general language capabilities in a package that remains practical to train and adapt on consumer-grade hardware. It is an excellent choice for custom fine-tuning projects where off-the-shelf chat behavior is not needed.

Qwen3 30B A3B Base

Alibaba · 30.5B · runs from 12.2 GB

Qwen3 30B A3B Base is a 30.5B-parameter open language model from Alibaba in the Qwen 3 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 3.1 8B

Meta · 8.0B · runs from 3.6 GB

Meta Llama 3.1 8B is an 8-billion parameter base (pretrained) model from the Llama 3.1 family. It is not instruction-tuned and is intended for fine-tuning, research, and custom downstream applications. Compared to Llama 3 8B, it extends the context window to 128K tokens and benefits from improved training data and methodology. The model uses grouped-query attention and was trained on a multilingual corpus. It is released under the Llama 3.1 Community License and is widely used as a foundation for community fine-tunes and specialized models.

Qwen3 0.6B Heretic Abliterated Uncensored

DavidAU · 596M · runs from 0.7 GB

Qwen3 0.6B Heretic Abliterated Uncensored is a 596M-parameter open language model from DavidAU in the Qwen 3 family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qianfan OCR

Baidu · 4.7B · runs from 2.5 GB

Qianfan-OCR is Baidu's 4.7-billion-parameter vision-language model for document intelligence rather than general chat. It pairs a Qianfan-ViT vision encoder with a Qwen3-4B language backbone, doing direct image-to-Markdown conversion alongside table extraction, chart understanding, and document Q&A in one end-to-end model instead of a multi-stage pipeline. It also has an optional "Layout-as-Thought" mode that reasons about page layout before producing output. At under 5 billion parameters, it runs on a single consumer or prosumer GPU once quantized. Context length is 32,768 tokens, extendable further per Baidu's documentation. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in March 2026, Baidu reports it as its top-scoring end-to-end model on public document-parsing benchmarks, supporting 192 languages.

OLMo 2 0325 32B Instruct

Allen AI · 32.2B · runs from 14.5 GB

OLMo 2 0325 32B Instruct is a 32.2B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

SmolLM 135M Instruct

Hugging Face · 135M · runs from 0.4 GB

SmolLM 135M Instruct is a 135M-parameter open language model from Hugging Face in the SmolLM family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

OneRec 1.7B

OpenOneRec · 2.1B · runs from 1.1 GB

OneRec 1.7B is a 2.1B-parameter open language model from OpenOneRec. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

LFM2 1.2B

Liquid AI · 1.2B · runs from 0.9 GB

LFM2 1.2B is a 1.2B-parameter open language model from Liquid AI in the LFM2 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3Guard Gen 0.6B

Alibaba · 752M · runs from 0.7 GB

Qwen3Guard Gen 0.6B is a 752M-parameter open language model from Alibaba in the Qwen 3 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

JiRackUltra 32B

CMSManhattan · 32.8B · runs from 9.8 GB

JiRackUltra 32B is a 32.8B-parameter open language model from CMSManhattan. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Kimi Linear 48B A3B Base

Moonshot AI · 49.1B · runs from 14.3 GB

Kimi Linear 48B A3B Base is a 49.1B-parameter open language model from Moonshot AI in the Kimi family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen1.5 0.5B Chat

Alibaba · 620M · runs from 0.8 GB

Qwen1.5 0.5B Chat is an early-generation small language model from Alibaba's Qwen series with just 620 million parameters. As one of the smallest models in the Qwen family, it was designed to demonstrate that useful conversational ability is possible even at sub-billion parameter scales. This model runs easily on virtually any hardware including CPUs, older GPUs, and even mobile devices. While its capabilities are limited compared to larger Qwen models, it remains a useful option for embedded applications, rapid prototyping, or situations where minimal resource consumption is the top priority.

Qwen3Guard Gen 4B

Alibaba · 4.4B · runs from 2.4 GB

Qwen3Guard Gen 4B is a 4.4B-parameter open language model from Alibaba in the Qwen 3 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.