All LLM Models
Browse 1242 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Molmo2 4B
Allen AI · 4.9B · runs from 2.5 GB
105.7K
54Molmo2-4B is Allen Institute for AI's (Ai2) vision-language model for image, video, and multi-image understanding and grounding, built on a Qwen3-4B-Instruct backbone with a SigLIP 2 vision encoder. It is trained on Ai2's own curated Molmo2 datasets rather than third-party captioning data of unclear provenance, and beyond ordinary visual question answering it supports pointing at and tracking objects across video frames. The card reports state-of-the-art results among open weight-and-data models on short-video understanding, counting, and captioning, with competitive results on long videos. At under 5 billion parameters, it fits on a single consumer GPU. Context length is 36,864 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2025.
Pythia 14M
EleutherAI · 14M · runs from 0.0 GB
Pythia 14M is a 14M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NousCoder 14B
Nous Research · 14.8B · runs from 6.9 GB
NousCoder 14B is a 14.8B-parameter open language model from Nous Research. It supports a context window of up to 81,920 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen 1 8B
Alibaba · 1.8B · runs from 0.9 GB
Qwen-1.8B is Alibaba's first-generation 1.8-billion-parameter base language model, pretrained from scratch on over 2.2 trillion tokens of Chinese, English, multilingual, code, and math data, with the same roughly 150,000-token vocabulary used across the Qwen family. It is a raw pretrained model rather than a chat assistant; Alibaba's aligned Qwen-1.8B-Chat is built on top of it. Its main selling point is low-cost deployment: the card reports int4/int8 quantized versions needing under 2GB of memory for inference and as little as 6GB for fine-tuning, so it runs comfortably on almost any consumer GPU or even a laptop. Context length is 8,192 tokens. It is released under a custom Tongyi Qianwen Research License, free for academic research, with commercial use requiring direct contact with Alibaba. It was published in November 2023.
Falcon 40B
TII UAE · 41.8B · runs from 19.6 GB
Falcon-40B is TII's 40-billion-parameter causal decoder-only base language model, pretrained from scratch on 1,000 billion tokens of the RefinedWeb dataset enhanced with curated corpora. At release TII described it as the best fully open model available, outperforming LLaMA, StableLM, and MPT on the Open LLM Leaderboard, with an inference-optimized architecture using FlashAttention and multi-query attention. It is a raw, pretrained model meant to be fine-tuned for chat or other downstream tasks rather than used directly; TII's own Falcon-40B-Instruct is the ready-to-use chat version built on top of it. Trained mainly on English, German, Spanish, and French, it needs a multi-GPU workstation in half precision, while a 4-bit quantization brings it within reach of a single high-memory GPU. License is Apache 2.0, permitting unrestricted commercial and research use. It was published in May 2023.
OLMo 1B HF
Allen AI · 1.2B · runs from 1.1 GB
OLMo 1B HF is a 1.2B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen 14B
Alibaba · 14.2B · runs from 6.6 GB
Qwen-14B is Alibaba's first-generation 14-billion-parameter base language model, pretrained from scratch on over 3 trillion tokens of Chinese, English, multilingual, code, and math data, with a roughly 150,000-token vocabulary for broader multilingual coverage. It is a raw pretrained Transformer, not tuned for conversation; Alibaba's aligned Qwen-14B-Chat is the assistant built on top of it. The card reports it beating other open models of similar size, and in some benchmarks larger models too, on Chinese and English evaluation suites. At 14B parameters it needs a capable consumer GPU at full precision, considerably less once quantized. Context length is 8,192 tokens, extendable further with the NTK-aware interpolation and window-attention techniques described in the card. It is released under a custom Tongyi Qianwen License Agreement that is free for research, with commercial use requiring a separate application to Alibaba. It was published in September 2023.
T5 Paraphrase Paws
Vamsi · 223M · runs from 0.1 GB
T5 Paraphrase Paws is a 223M-parameter open language model from Vamsi. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 3 Mini 128k Instruct
Microsoft · 3.8B · runs from 2.7 GB
Phi 3 Mini 128k Instruct is a 3.8B-parameter open language model from Microsoft in the Phi 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Sarvam 30B Uncensored
aoxo · 32.2B · runs from 14 GB
Sarvam 30B Uncensored is a 32.2B-parameter open language model from aoxo. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 12B IT Assistant
Google · 12B · runs from 5.4 GB
Gemma 4 12B IT Assistant is a 12B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi Tiny MoE Instruct
Microsoft · 3.8B · runs from 2.2 GB
Phi Tiny MoE Instruct is a 3.8B-parameter open language model from Microsoft in the Phi family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron Labs Diffusion 3B
NVIDIA · 3.8B · runs from 2.1 GB
Nemotron Labs Diffusion 3B is a 3.8B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
A.X 4.0 Light
skt · 7.3B · runs from 2.4 GB
A.X 4.0 Light is a 7.3B-parameter open language model from skt. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Tiny Mixtral
TitanML · 247M · runs from 0.4 GB
Tiny Mixtral is a 247M-parameter open language model from TitanML in the Mixtral family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 3.1 Nemotron Nano 8B V1
NVIDIA · 8B · runs from 2.8 GB
Llama 3.1 Nemotron Nano 8B is an 8-billion parameter chat model by NVIDIA, a compact entry in the Nemotron family derived from Meta's Llama 3.1 architecture. It applies NVIDIA's alignment and fine-tuning techniques to deliver improved response quality over the base Llama 3.1 8B Instruct model at the same parameter count. The model runs on consumer GPUs with 8GB or more of VRAM and supports a 128K token context window. Its small footprint and NVIDIA-tuned quality make it a practical option for local inference on mainstream hardware.
Baichuan 13B Base
baichuan-inc · 13B · runs from 6.1 GB
Baichuan-13B-Base is Baichuan Intelligence's 13-billion-parameter pretrained base model, the follow-up to Baichuan-7B, trained on 1.4 trillion tokens of bilingual Chinese/English data, about 40% more than LLaMA-13B at the time. It replaces the usual rotary position embeddings with ALiBi linear position biasing, which the authors report gives roughly 31.6% faster token generation than a comparable LLaMA-13B. Official int8 and int4 quantized versions are provided so the model deploys on consumer GPUs such as the RTX 3090 with little accuracy loss. A separately released Baichuan-13B-Chat provides the aligned, conversational counterpart to this base checkpoint. Context length is 4,096 tokens. It is released under a custom Community License for the Baichuan-13B Model: free for academic research, with commercial use requiring written authorization from Baichuan via email request. It was published in July 2023, an early bilingual open model that predates the Baichuan2 series.
Baichuan2 13B Base
baichuan-inc · 13B · runs from 6.1 GB
Baichuan2-13B-Base is Baichuan Intelligence's 13-billion-parameter base language model — pretrained, not instruction-tuned — from the second-generation Baichuan series, trained on 2.6 trillion tokens of high-quality Chinese and English text. At release it topped same-size open models on Chinese and English benchmarks spanning general knowledge, law, medicine, math, code, and multilingual translation; a separate Baichuan2-13B-Chat model built on top of it adds instruction and safety tuning. As a 13-billion-parameter dense model, it fits on a single consumer GPU once quantized. It is released under a custom Baichuan2 community license: free for academic research and for commercial use once a developer applies for and receives a free commercial license from Baichuan by email. It was published in September 2023.
GLM 4.7 Flash Coder
whitecircle · 29.9B · runs from 13.8 GB
GLM 4.7 Flash Coder is a 29.9B-parameter open language model from whitecircle in the GLM 4 family. It supports a context window of up to 202,752 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.5 4B
Alibaba · 4.7B · runs from 2.5 GB
Qwen3.5-4B is Alibaba's dense 4-billion-parameter vision-language model from the Qwen3.5 generation, sharing the same hybrid Gated DeltaNet and gated-attention architecture as its larger siblings. This is the post-trained, instruction-tuned release, able to process images together with text and handle general chat, coding assistance, and agentic tool use. It has a 262,144-token native context window, extensible up to roughly 1,010,000 tokens, and is released under the Apache 2.0 license. At around 4.7 billion parameters, 4-bit quantization needs under 3GB of memory, so it runs easily on almost any modern GPU or laptop.
Opt 125M
Meta · 125M · runs from 0.3 GB
Meta OPT 125M is a 125-million parameter language model from Meta's Open Pre-trained Transformer (OPT) project. Released in 2022, it was part of Meta's effort to provide the research community with openly available large language models that replicate the performance of GPT-3 class models at various scales. As one of the smallest models in the OPT family, the 125M variant is primarily useful for research, experimentation, and educational purposes. It can run on virtually any hardware, including CPU-only setups. While significantly less capable than modern models, it remains a useful reference point in LLM research.
OTel 2.0 LLM 31B IT
farbodtavakkoli · 31.3B · runs from 14.9 GB
OTel 2.0 LLM 31B IT is a 31.3B-parameter open language model from farbodtavakkoli. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.5 35B A3B
Alibaba · 36.0B · runs from 15.7 GB
Qwen3.5 35B A3B is Alibaba's mixture-of-experts vision-language model in the Qwen3.5 line, totaling 36 billion parameters with about 3 billion active per token (the A3B in its name). It accepts text, images, and video as input, using a hybrid linear-attention and sparse-MoE architecture built for efficiency. Only the active parameters compute per token, so generation is faster than a same-sized dense model, though all 36 billion weights must fit in memory, calling for a high-end 24-32GB-class GPU once quantized. The model offers a 262K token context window for long documents or multi-turn chats. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2026 as part of Alibaba's push toward efficient multimodal MoE models.
Llava 1.5 7B HF
llava-hf · 7.1B · runs from 3.3 GB
LLaVA-1.5-7B is a 7-billion-parameter vision-language chat model made by fine-tuning Vicuna-7B, itself based on Llama 2, on GPT-generated multimodal instruction data, paired with a CLIP ViT-L/14 vision encoder via an MLP projector. It answers questions about images, describes visual content, and follows multi-turn multimodal instructions, and was one of the first widely used open vision-language chat models. At 7B it runs comfortably on a single consumer GPU once quantized. Context length is limited to 4,096 tokens, short by current standards given its 2023-era Llama 2 backbone. It is released under the Llama 2 Community License, permitting commercial use with some restrictions, including a separate license for very large companies; published in December 2023. Unlike newer small vision-language models, LLaVA-1.5 uses a simple architecture with no token-compression for image patches.
OTel LLM E4B IT
farbodtavakkoli · 4B · runs from 8.5 GB
OTel LLM E4B IT is a 4B-parameter open language model from farbodtavakkoli. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
H2ovl Mississippi 2B
h2oai · 2.2B · runs from 4.7 GB
H2OVL-Mississippi-2B is H2O.ai's 2.15-billion-parameter vision-language model, extending their H2O-Danube2-1.8B chat model with a vision encoder for multimodal tasks. It handles image captioning, visual question answering, and document understanding, positioned as an efficient, general-purpose OCR and document-AI model. H2O.ai trained it on 17 million image-text pairs and reports it competing with other 2B-class models like Qwen2-VL-2B and InternVL2-2B on public benchmarks. At this size, it runs easily on a single consumer GPU, even unquantized. Context length is 8,192 tokens, inherited from its Danube2 language backbone. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2024, it is part of H2O.ai's Mississippi family of open document-AI and OCR-focused vision-language models.
OTel LLM 27B IT
farbodtavakkoli · 27B · runs from 13.1 GB
OTel LLM 27B IT is a 27B-parameter open language model from farbodtavakkoli. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 2 7B HF
Meta · 6.7B · runs from 3.1 GB
Meta Llama 2 7B is a 6.7-billion parameter base (pretrained) language model from Meta's Llama 2 generation, provided in Hugging Face Transformers format. It was trained on 2 trillion tokens with a 4K token context window and represented a significant step in openly available large language models when released. As a base model, it is designed for further fine-tuning and research rather than direct chat use. While superseded by Llama 3 and later releases in terms of benchmark performance, Llama 2 7B remains widely used in the research community and as a baseline for comparison. Released under the Llama 2 Community License.
Phi 3.5 Vision Instruct
Microsoft · 4.1B · runs from 2.9 GB
Phi 3.5 Vision Instruct is Microsoft's 4.1-billion-parameter multimodal model, pairing an image encoder and connector with the Phi-3 Mini language model to handle text and code alongside pictures. It suits visual question answering, chart and table reading, document OCR, and comparing details across multiple images in one prompt. Its small size suits laptops and modest consumer GPUs, running smoothly even on limited hardware once quantized. The model supports a 128K token context window, enough for lengthy documents. It is released under the MIT license, one of the most permissive options available, allowing unrestricted commercial and research use. Published in August 2024, it was trained on roughly 500 billion tokens of synthetic and filtered web data.
InternVL2 2B
OpenGVLab · 2.2B · runs from 4.8 GB
InternVL2-2B is OpenGVLab's 2.2-billion-parameter instruction-tuned vision-language model, part of the InternVL 2.0 family that ranges from 1 billion to 108 billion parameters. It pairs the InternViT-300M-448px vision encoder with the internlm2-chat-1.8b language model through an MLP projector, and is trained to handle document and chart comprehension, infographics QA, scene-text and OCR tasks, scientific and mathematical problem solving, and multi-image or video input. Compared with earlier Mini-InternVL models it adds support for long texts, multiple images, and video within the same training context. At just over 2 billion parameters, it runs comfortably on a single consumer GPU or even weaker hardware. Context length is 8,192 tokens, matching its 8k training context window. It is released under the MIT license, permitting unrestricted commercial and research use. It was published in June 2024, and has since been superseded by the InternVL2.5 and InternVL3 series.