All LLM Models
Browse 982 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Unlimited OCR
Baidu · 3.3B · runs from 1.6 GB
2.1M
4.3KUnlimited-OCR is Baidu's 3.3-billion-parameter vision-language model for OCR and document parsing, built to extend DeepSeek-OCR's approach further. It pairs a vision encoder with a Mixture-of-Experts decoder on DeepSeek's architecture, routing to 6 of 64 experts per token plus 2 shared experts, so only about 1.1 billion parameters activate per token even though every expert must still fit in memory. Its distinguishing idea, "one-shot long-horizon parsing," extends optical context compression to longer documents processed in a single pass. It is small enough to run on a single consumer GPU once quantized. Context length is 32,768 tokens, oriented around page and document parsing. It is released under the MIT license, a highly permissive option for commercial use. Published in June 2026, it has quickly picked up community support for vLLM inference and ms-swift fine-tuning.
Qwen2 0.5B Instruct
Alibaba · 494M · runs from 0.5 GB
Qwen2 0.5B Instruct is a 494M-parameter open language model from Alibaba in the Qwen 2 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 7B Instruct 1M
Alibaba · 7.6B · runs from 3.0 GB
Qwen2.5 7B Instruct 1M is a 7.6B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 1,010,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Hermes 4 14B
Nous Research · 14.8B · runs from 5.1 GB
Hermes 4 14B is a 14.8B-parameter open language model from Nous Research in the Hermes family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Magistral Small 2509
Mistral AI · 24.0B · runs from 7.3 GB
Magistral Small 2509 (also called Magistral Small 1.2) is Mistral AI's small reasoning model, built on Mistral Small 3.2 24B Instruct with added chain-of-thought reasoning trained through supervised fine-tuning on Magistral Medium traces followed by reinforcement learning. Unlike the text-only Magistral Small 1.1, this 1.2 release adds a vision encoder, so it can reason over images as well as text, wrapping its reasoning trace in dedicated [THINK]/[/THINK] tokens. It supports dozens of languages and is small enough to fit on a single consumer GPU once quantized. Context length is 131,072 tokens, though the card notes performance may degrade somewhat past 40,000 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2025.
Gemma 2 27B IT
Google · 27.2B · runs from 9.0 GB
Google Gemma 2 27B IT is a 27.2-billion parameter instruction-tuned model from Google's Gemma 2 generation. It is a text-only chat model optimized for conversational use, reasoning, and instruction following. Gemma 2 27B IT was one of the strongest openly available models in its size class at release. The model requires a GPU with at least 24GB of VRAM for quantized local inference. It is widely supported by popular inference engines and remains a strong choice for users seeking high-quality local chat without needing 70B-class hardware. Released under the Gemma license.
Hermes 3 Llama 3.1 8B
Nous Research · 8.0B · runs from 3.3 GB
Hermes 3 Llama 3.1 8B is an 8-billion parameter instruction-tuned model by Nous Research, built on Meta's Llama 3.1 8B base. It is fine-tuned for advanced instruction following, multi-turn conversation, structured output, and creative roleplay scenarios. The Hermes series is known for producing highly steerable models that respond well to system prompts. This model supports a 128K token context window inherited from the Llama 3.1 architecture and runs efficiently on consumer GPUs with 8GB or more of VRAM. It is a popular choice among local inference enthusiasts who value strong instruction adherence and versatile conversational ability.
LFM2 VL 1.6B
Liquid AI · 1.6B · runs from 0.8 GB
LFM2-VL-1.6B is Liquid AI's 1.6-billion-parameter vision-language model, pairing a hybrid convolution-and-attention LFM2 language backbone (1.2B parameters) with a 400-million-parameter SigLIP2 NaFlex vision encoder. It targets image understanding and lightweight agentic tasks with variable-resolution inputs, tuned for low latency over heavy reasoning; Liquid AI recommends fine-tuning it for narrow use cases. It is small enough to run on a single modest consumer GPU. The model supports a 128,000 token context window. It is released under Liquid AI's LFM Open License v1.0, free including for commercial use under $10 million in annual revenue, above which a separate license applies. Published in August 2025 alongside a smaller 450M sibling, it claims roughly 2x faster GPU inference than comparable models while handling native 512x512 resolution without upscaling.
SmolLM2 1.7B Instruct
Hugging Face · 1.7B · runs from 1.4 GB
SmolLM2 1.7B Instruct is the largest instruction-tuned model in the SmolLM2 family, offering the best balance of capability and efficiency Hugging Face achieved with this generation. At 1.7 billion parameters it produces substantially more coherent and useful responses than its smaller siblings, handling multi-turn conversations, summarization, and simple reasoning tasks with competence. With VRAM requirements well under 4 GB at standard precision, this model runs effortlessly on entry-level GPUs, older laptops, and even some mobile devices. It is an excellent choice for developers building lightweight local assistants or chatbots who want genuine conversational quality without the hardware demands of larger models.
Qwen2.5 3B
Alibaba · 3.1B · runs from 1.6 GB
Qwen2.5 3B is a 3.1B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 E4B IT Assistant
Google · 4B · runs from 2 GB
Gemma 4 E4B IT Assistant is a 4B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
MiniCPM5 1B Agentic Tooluse Merged FP16
ewinregirgojr · 1.1B · runs from 0.8 GB
MiniCPM5 1B Agentic Tooluse Merged FP16 is a 1.1B-parameter open language model from ewinregirgojr in the MiniCPM family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 VL 2B Thinking
Alibaba · 2.1B · runs from 1.1 GB
Qwen3-VL-2B-Thinking is Alibaba's 2.1-billion-parameter vision-language model, the reasoning-enhanced "Thinking" edition of the smallest Qwen3-VL checkpoint, alongside a matching Instruct edition. Beyond image description and visual question answering, it acts as a visual agent capable of operating PC and mobile GUIs, generating code from screenshots or diagrams, and reasoning about 2D and 3D spatial relationships. Its OCR pipeline covers 32 languages and is tuned for low light, blur, and tilted text. At just over 2 billion parameters, it runs easily on a single modest consumer GPU, even unquantized. The model natively supports a 262,144 token context window, expandable to 1,010,000 tokens for long documents or hours of video. It is released under the Apache 2.0 license, and was published in October 2025. Compared with Qwen2.5-VL, Qwen3-VL adds DeepStack vision-feature fusion and timestamp-grounded video event localization.
Llama 3.2 11B Vision Instruct
Meta · 10.7B · runs from 5.0 GB
Llama 3.2 11B Vision Instruct is a smaller vision-language model in Meta's Llama 3 family, with roughly 11 billion parameters, built to process images together with text and follow chat-style instructions. It brings multimodal capability, such as visual question answering and image description, to a size that is far more approachable than the 90-billion-parameter variant from the same release. Released in September 2024 under Meta's Llama 3.2 community license, it does not carry a fully permissive open license but does allow broad use under Meta's terms. At around 11 billion parameters, it fits comfortably on a single mid-range to high-end consumer GPU once quantized to 4-bit, putting it within reach of enthusiast local setups.
OLMoE 1B 7B 0924 Instruct
Allen AI · 6.9B · runs from 3.5 GB
OLMoE 1B 7B 0924 Instruct is a 6.9B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nanonets OCR S
nanonets · 3.8B · runs from 1.4 GB
Nanonets-OCR-s is Nanonets' image-to-markdown OCR model, built on top of Qwen2.5-VL-3B-Instruct, that goes beyond plain text extraction to produce structured markdown for downstream processing by other language models. It converts mathematical equations and formulas into LaTeX, describes embedded images and charts inside structured tags, isolates signatures and watermarks into their own tags, converts checkboxes into standard Unicode symbols, and extracts complex tables into both markdown and HTML formats. At under 4 billion parameters it is light enough to run on a single consumer GPU. Context length is 128,000 tokens. Nanonets has not published a license for the model on its Hugging Face model card. It was published in June 2025.
Granite 4.1 30B
IBM · 28.9B · runs from 8.8 GB
Granite 4.1 30B is IBM's 28.9-billion-parameter dense language model, tuned for chat, tool calling, and general assistant tasks as part of the Granite 4.1 family alongside smaller 3B and 8B variants. IBM built the 4.1 generation to be enterprise-friendly, with a focus on reliable tool calling and instruction following for agentic workflows. At this size, local inference calls for quantization and a capable GPU, fitting on a single high-end consumer or workstation card. The model supports a 131,072 token context window, enough for lengthy documents, codebases, or long multi-turn conversations. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in April 2026, it is the largest model in the Granite 4.1 generation, which succeeds Granite 4.0.
Gemma 4 E4B IT Uncensored
TrevorJS · 8.0B · runs from 3.9 GB
Gemma 4 E4B IT Uncensored is a 8.0B-parameter open language model from TrevorJS in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Olmo 3.1 32B Instruct
Allen AI · 32.2B · runs from 9.7 GB
Olmo-3.1-32B-Instruct is Allen Institute for AI's 32-billion-parameter open instruction-tuned language model, part of the Olmo 3 family that also includes 7B and 32B "Think" reasoning variants. It is pretrained on the Dolma 3 corpus and post-trained in three stages on the Dolci datasets, covering math, code, chat, and instruction-following: supervised fine-tuning, direct preference optimization, and reinforcement learning from verifiable rewards. Ai2 reports it competitive with or ahead of similarly sized open models such as Qwen 3 32B and Gemma 3 27B on math, coding, and reasoning benchmarks, and it needs a multi-GPU workstation to run at full precision, considerably less once quantized. Context length is 65,536 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2025, a point release built on the same Olmo-3-32B base as the original Olmo 3 family.
Granite 4.0 H Small
IBM · 32.2B · runs from 9.5 GB
Granite-4.0-H-Small is IBM's 32-billion-parameter instruction-tuned model in the Granite 4.0 line, fine-tuned from Granite-4.0-H-Small-Base for enterprise use: summarization, classification, extraction, question answering, retrieval-augmented generation, code tasks, function calling, and multilingual dialog. Architecturally it is a hybrid mixture-of-experts model with only 4 full-attention layers against 36 Mamba-2 state-space layers, and 72 experts with 10 active per token, giving about 9 billion active parameters out of 32 billion total; the Mamba-2 layers avoid growing a KV cache with context length, keeping long-context inference cheaper than a pure attention stack. It was trained with supervised fine-tuning, RL-based alignment, and model merging. At 32 billion parameters, it needs a high-end consumer GPU or multi-GPU setup once quantized. Context length is 131,072 tokens (128K). It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in October 2025.
HunyuanOCR
Tencent · 1.1B · runs from 0.7 GB
HunyuanOCR is Tencent's 1.1-billion-parameter vision-language model built for OCR and document understanding, unifying document parsing, text spotting, information extraction, and text-image translation in one model rather than a general chat assistant. The current checkpoint, HunyuanOCR-1.5, adds DFlash speculative decoding, where a lightweight draft model proposes tokens that the main model verifies in one pass, cutting latency on long structured outputs like tables and formulas. It also ships GGUF weights for llama.cpp, running on CPUs and laptop or consumer GPUs, not just server-grade vLLM. At just over 1 billion parameters, it is light enough for a single modest consumer GPU. Context length is 131,072 tokens; pretraining supports image resolutions up to 4K. It is released under the Tencent Hunyuan Community License, a custom license permitting commercial use outside the EU, UK, and South Korea and below 100 million monthly active users. It was published in November 2025.
Qwen2 VL 7B Instruct
Alibaba · 8.3B · runs from 3.2 GB
Qwen2 VL 7B Instruct is Alibaba's 8.3-billion-parameter vision-language model from the original Qwen 2 generation, built to process images and video alongside text in a single conversation. It can describe images, answer visual questions, and reason over multi-image or video input, suiting it to multimodal chat and visual document tasks. At this parameter count, local inference is practical on a single mainstream or high-end consumer GPU once the weights are quantized, rather than requiring multi-GPU hardware. The model supports a 32K token context window, enough for moderate-length documents or multi-turn visual conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in August 2024, it was later superseded by Qwen2.5-VL-7B-Instruct, which its own model card lists as its successor.
SmolVLM 256M Instruct
Hugging Face · 256M · runs from 0.4 GB
SmolVLM-256M Instruct is Hugging Face's 256-million-parameter vision-language model, built on a compact SmolLM2-135M language backbone paired with a small 93-million-parameter SigLIP vision encoder. Hugging Face calls it the smallest publicly available multimodal model, designed for image captioning, visual question answering, and basic text transcription from images rather than open-ended chat. Its size makes it practical to run even on CPUs or entry-level GPUs, and on-device deployment is a stated use case. Context length is limited to 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in January 2025. Compared to the larger 2.2B SmolVLM2 sibling, it uses a smaller vision encoder and more aggressive image-token compression to further cut memory and latency, at some cost to accuracy.
Granite 4.0 Micro
IBM · 3.4B · runs from 1.4 GB
Granite-4.0-Micro is IBM's 3-billion-parameter long-context instruct model, fine-tuned from Granite-4.0-Micro-Base on a mix of permissively licensed open datasets and IBM's own synthetic data, using supervised fine-tuning, reinforcement-learning alignment, and model merging. Unlike some of its Granite 4.0 siblings (H Micro, H Tiny MoE, H Small MoE), which mix in Mamba2 layers or mixture-of-experts routing, Granite-4.0-Micro is a plain decoder-only dense transformer with grouped-query attention, RoPE, SwiGLU, and RMSNorm. It targets enterprise use cases such as summarization, RAG, tool-calling, and code completion, and at 3B parameters it runs comfortably on a single consumer GPU, even quantized on a laptop. Context length is 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in October 2025.
LFM2.5 1.2B Thinking
Liquid AI · 1.2B · runs from 0.9 GB
LFM2.5 1.2B Thinking is a 1.2B-parameter open language model from Liquid AI in the LFM2.5 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
K2 Horizon 7B
IFM · 9.0B · runs from 4.4 GB
K2 Horizon 7B is a 9.0B-parameter open language model from IFM. It supports a context window of up to 524,288 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mistral 7B v0.1
Mistral AI · 7.2B · runs from 3.6 GB
Mistral 7B v0.1 is the original base model from Mistral AI that helped reshape expectations for small open-weight language models when it launched in late 2023. As a pretrained foundation model without instruction tuning, it is designed for fine-tuning, research, and custom downstream tasks rather than direct conversational use. With 7 billion parameters and support for grouped-query attention and sliding-window attention, it remains a popular starting point for practitioners building specialized models. Its modest VRAM requirements of roughly 6 GB at 4-bit quantization keep it accessible on a wide range of consumer GPUs.
DeepSeek R1 Distill Qwen 14B Abliterated v2
huihui-ai · 14.8B · runs from 7.0 GB
DeepSeek R1 Distill Qwen 14B Abliterated v2 is a 14.8B-parameter open language model from huihui-ai in the DeepSeek R1 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Granite 3.3 8B Instruct
IBM · 8.2B · runs from 2.9 GB
Granite 3.3 8B Instruct is a 8.2B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Chandra Ocr 2
datalab-to · 5.3B · runs from 2.7 GB
Chandra OCR 2 is Datalab's 5.3-billion-parameter vision-language model for OCR and document conversion, turning scanned pages and PDFs into markdown, HTML, or JSON while preserving layout. It runs on a Qwen3.5-based backbone mixing linear attention with periodic full-attention layers, and handles handwriting, forms, tables, and math across 90-plus languages. It is the second generation of Datalab's Chandra model, improved over the first release. At just over 5 billion parameters, it runs on a single consumer GPU once quantized. Context extends to 262,144 tokens, generous for multi-page documents. It carries a modified OpenRAIL-M license: free for research, personal use, and startups under $2 million in funding or revenue, but it bars competing against Datalab's own API, more restrictive than a standard permissive license. Published in March 2026, following the original Chandra.