All LLM Models

Browse 1242 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Gemma 4 26B A4B IT Assistant

Google · 26B · runs from 11.4 GB

Gemma 4 26B A4B IT Assistant is a 26B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen2.5 Coder 32B

Alibaba · 32.8B · runs from 9.8 GB

Qwen2.5-Coder-32B is Alibaba's 32.8-billion-parameter code-specialized base language model, the largest of six sizes in the Qwen2.5-Coder family (0.5B to 32B), continued-pretrained from Qwen2.5-32B on 5.5 trillion tokens of source code, text-code grounding data, and synthetic data. It is a raw pretrained model, not instruction-tuned, so it is meant as a foundation for building code assistants, agents, or chat models through further fine-tuning rather than for direct conversational use. Alibaba states its coding ability rivals GPT-4o, and at this size it needs a high-end consumer GPU or multi-GPU setup once quantized, more at full precision. Context length defaults to 32,768 tokens, extendable to 131,072 tokens with YaRN scaling. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in November 2024.

LFM2 8B A1B

Liquid AI · 8.3B · runs from 2.7 GB

LFM2 8B A1B is Liquid AI's larger mixture-of-experts model, combining the company's novel hybrid architecture with approximately 8 billion total parameters. It uses a MoE design to keep active compute per token low while maintaining strong general performance across chat and reasoning tasks. For local users, it offers an intriguing alternative to conventional 8B transformers, with Liquid AI's architecture promising improved efficiency and throughput on consumer-grade hardware.

DeepSeek Coder v2 Lite Instruct

DeepSeek · 15.7B · runs from 7.2 GB

DeepSeek Coder V2 Lite Instruct is a code-focused mixture-of-experts model with 15.7 billion total parameters, trained to handle both programming tasks and general conversation. It supports a wide range of programming languages and excels at code generation, debugging, explanation, and refactoring. The MoE architecture keeps compute costs manageable despite the model's broad capabilities, and the Lite variant is sized to run on a single consumer GPU. For developers looking for a capable local coding assistant that can also handle general chat, this model offers an appealing combination of code specialization and practical hardware requirements.

Qwen3 Omni 30B A3B Thinking

Alibaba · 31.7B · runs from 14.8 GB

Qwen3-Omni-30B-A3B-Thinking is the reasoning-focused variant of Alibaba's Qwen3-Omni family, a mixture-of-experts model that accepts text, audio, image, and video input and reasons over it with explicit chain-of-thought before answering in text. Unlike the Instruct variant, it keeps only the "thinker" component and drops the speech-generating "talker" and audio codec, so it outputs text only, not speech. Its backbone still routes across 128 experts with 8 active per token, roughly 3 billion active out of about 31.7 billion total, and needs a high-end consumer GPU or multi-GPU setup once quantized. Context length is 65,536 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2025, a few days ahead of the Instruct variant that adds real-time speech generation on top of the same reasoning core.

SmolVLM2 500M Video Instruct

Hugging Face · 507M · runs from 0.5 GB

SmolVLM2-500M Video Instruct is Hugging Face's roughly 507-million-parameter vision-language model, a mid-sized member of the SmolVLM2 family purpose-built for analyzing video alongside images and text. It can answer questions about a clip, compare visual content across frames, or transcribe on-screen text, and Hugging Face designed it specifically for on-device video understanding where compute is limited. Its modest size makes it comfortable to run on a single consumer GPU. Context length is 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2025 alongside the larger 2.2B and smaller 256M SmolVLM2 models. Unlike the image-focused 256M model, both the 500M and 2.2B variants are explicitly trained and named for video understanding.

Jan Code 4B

janhq · 4.4B · runs from 2.4 GB

Jan Code 4B is a 4.4B-parameter open language model from janhq. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatFunctionsCode

Unlimited OCR

Baidu · 3.3B · runs from 1.6 GB

Unlimited-OCR is Baidu's 3.3-billion-parameter vision-language model for OCR and document parsing, built to extend DeepSeek-OCR's approach further. It pairs a vision encoder with a Mixture-of-Experts decoder on DeepSeek's architecture, routing to 6 of 64 experts per token plus 2 shared experts, so only about 1.1 billion parameters activate per token even though every expert must still fit in memory. Its distinguishing idea, "one-shot long-horizon parsing," extends optical context compression to longer documents processed in a single pass. It is small enough to run on a single consumer GPU once quantized. Context length is 32,768 tokens, oriented around page and document parsing. It is released under the MIT license, a highly permissive option for commercial use. Published in June 2026, it has quickly picked up community support for vLLM inference and ms-swift fine-tuning.

Qwen2 0.5B Instruct

Alibaba · 494M · runs from 0.5 GB

Qwen2 0.5B Instruct is a 494M-parameter open language model from Alibaba in the Qwen 2 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen2.5 7B Instruct 1M

Alibaba · 7.6B · runs from 3.0 GB

Qwen2.5 7B Instruct 1M is a 7.6B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 1,010,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Hermes 4 14B

Nous Research · 14.8B · runs from 5.1 GB

Hermes 4 14B is a 14.8B-parameter open language model from Nous Research in the Hermes family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatReasoningRoleplay

Magistral Small 2509

Mistral AI · 24.0B · runs from 7.3 GB

Magistral Small 2509 (also called Magistral Small 1.2) is Mistral AI's small reasoning model, built on Mistral Small 3.2 24B Instruct with added chain-of-thought reasoning trained through supervised fine-tuning on Magistral Medium traces followed by reinforcement learning. Unlike the text-only Magistral Small 1.1, this 1.2 release adds a vision encoder, so it can reason over images as well as text, wrapping its reasoning trace in dedicated [THINK]/[/THINK] tokens. It supports dozens of languages and is small enough to fit on a single consumer GPU once quantized. Context length is 131,072 tokens, though the card notes performance may degrade somewhat past 40,000 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2025.

Gemma 2 27B IT

Google · 27.2B · runs from 9.0 GB

Google Gemma 2 27B IT is a 27.2-billion parameter instruction-tuned model from Google's Gemma 2 generation. It is a text-only chat model optimized for conversational use, reasoning, and instruction following. Gemma 2 27B IT was one of the strongest openly available models in its size class at release. The model requires a GPU with at least 24GB of VRAM for quantized local inference. It is widely supported by popular inference engines and remains a strong choice for users seeking high-quality local chat without needing 70B-class hardware. Released under the Gemma license.

Hermes 3 Llama 3.1 8B

Nous Research · 8.0B · runs from 3.3 GB

Hermes 3 Llama 3.1 8B is an 8-billion parameter instruction-tuned model by Nous Research, built on Meta's Llama 3.1 8B base. It is fine-tuned for advanced instruction following, multi-turn conversation, structured output, and creative roleplay scenarios. The Hermes series is known for producing highly steerable models that respond well to system prompts. This model supports a 128K token context window inherited from the Llama 3.1 architecture and runs efficiently on consumer GPUs with 8GB or more of VRAM. It is a popular choice among local inference enthusiasts who value strong instruction adherence and versatile conversational ability.

LFM2 VL 1.6B

Liquid AI · 1.6B · runs from 0.8 GB

LFM2-VL-1.6B is Liquid AI's 1.6-billion-parameter vision-language model, pairing a hybrid convolution-and-attention LFM2 language backbone (1.2B parameters) with a 400-million-parameter SigLIP2 NaFlex vision encoder. It targets image understanding and lightweight agentic tasks with variable-resolution inputs, tuned for low latency over heavy reasoning; Liquid AI recommends fine-tuning it for narrow use cases. It is small enough to run on a single modest consumer GPU. The model supports a 128,000 token context window. It is released under Liquid AI's LFM Open License v1.0, free including for commercial use under $10 million in annual revenue, above which a separate license applies. Published in August 2025 alongside a smaller 450M sibling, it claims roughly 2x faster GPU inference than comparable models while handling native 512x512 resolution without upscaling.

SmolLM2 1.7B Instruct

Hugging Face · 1.7B · runs from 1.4 GB

SmolLM2 1.7B Instruct is the largest instruction-tuned model in the SmolLM2 family, offering the best balance of capability and efficiency Hugging Face achieved with this generation. At 1.7 billion parameters it produces substantially more coherent and useful responses than its smaller siblings, handling multi-turn conversations, summarization, and simple reasoning tasks with competence. With VRAM requirements well under 4 GB at standard precision, this model runs effortlessly on entry-level GPUs, older laptops, and even some mobile devices. It is an excellent choice for developers building lightweight local assistants or chatbots who want genuine conversational quality without the hardware demands of larger models.

Qwen2.5 3B

Alibaba · 3.1B · runs from 1.6 GB

Qwen2.5 3B is a 3.1B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 4 E4B IT Assistant

Google · 4B · runs from 2 GB

Gemma 4 E4B IT Assistant is a 4B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MiniCPM5 1B Agentic Tooluse Merged FP16

ewinregirgojr · 1.1B · runs from 0.8 GB

MiniCPM5 1B Agentic Tooluse Merged FP16 is a 1.1B-parameter open language model from ewinregirgojr in the MiniCPM family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3 VL 2B Thinking

Alibaba · 2.1B · runs from 1.1 GB

Qwen3-VL-2B-Thinking is Alibaba's 2.1-billion-parameter vision-language model, the reasoning-enhanced "Thinking" edition of the smallest Qwen3-VL checkpoint, alongside a matching Instruct edition. Beyond image description and visual question answering, it acts as a visual agent capable of operating PC and mobile GUIs, generating code from screenshots or diagrams, and reasoning about 2D and 3D spatial relationships. Its OCR pipeline covers 32 languages and is tuned for low light, blur, and tilted text. At just over 2 billion parameters, it runs easily on a single modest consumer GPU, even unquantized. The model natively supports a 262,144 token context window, expandable to 1,010,000 tokens for long documents or hours of video. It is released under the Apache 2.0 license, and was published in October 2025. Compared with Qwen2.5-VL, Qwen3-VL adds DeepStack vision-feature fusion and timestamp-grounded video event localization.

Llama 3.2 11B Vision Instruct

Meta · 10.7B · runs from 5.0 GB

Llama 3.2 11B Vision Instruct is a smaller vision-language model in Meta's Llama 3 family, with roughly 11 billion parameters, built to process images together with text and follow chat-style instructions. It brings multimodal capability, such as visual question answering and image description, to a size that is far more approachable than the 90-billion-parameter variant from the same release. Released in September 2024 under Meta's Llama 3.2 community license, it does not carry a fully permissive open license but does allow broad use under Meta's terms. At around 11 billion parameters, it fits comfortably on a single mid-range to high-end consumer GPU once quantized to 4-bit, putting it within reach of enthusiast local setups.

Qwen3 30B A3B Abliterated

mlabonne · 30.5B · runs from 13.4 GB

Qwen3 30B A3B Abliterated is a 30.5B-parameter open language model from mlabonne in the Qwen 3 family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

OLMoE 1B 7B 0924 Instruct

Allen AI · 6.9B · runs from 3.5 GB

OLMoE 1B 7B 0924 Instruct is a 6.9B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Ornith 1.0 35B Uncensored Heretic

llmfan46 · 35.1B · runs from 15.3 GB

Ornith 1.0 35B Uncensored Heretic is a 35.1B-parameter open language model from llmfan46 in the Ornith family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Nanonets OCR S

nanonets · 3.8B · runs from 1.4 GB

Nanonets-OCR-s is Nanonets' image-to-markdown OCR model, built on top of Qwen2.5-VL-3B-Instruct, that goes beyond plain text extraction to produce structured markdown for downstream processing by other language models. It converts mathematical equations and formulas into LaTeX, describes embedded images and charts inside structured tags, isolates signatures and watermarks into their own tags, converts checkboxes into standard Unicode symbols, and extracts complex tables into both markdown and HTML formats. At under 4 billion parameters it is light enough to run on a single consumer GPU. Context length is 128,000 tokens. Nanonets has not published a license for the model on its Hugging Face model card. It was published in June 2025.

Granite 4.1 30B

IBM · 28.9B · runs from 8.8 GB

Granite 4.1 30B is IBM's 28.9-billion-parameter dense language model, tuned for chat, tool calling, and general assistant tasks as part of the Granite 4.1 family alongside smaller 3B and 8B variants. IBM built the 4.1 generation to be enterprise-friendly, with a focus on reliable tool calling and instruction following for agentic workflows. At this size, local inference calls for quantization and a capable GPU, fitting on a single high-end consumer or workstation card. The model supports a 131,072 token context window, enough for lengthy documents, codebases, or long multi-turn conversations. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in April 2026, it is the largest model in the Granite 4.1 generation, which succeeds Granite 4.0.

Gemma 4 E4B IT Uncensored

TrevorJS · 8.0B · runs from 3.9 GB

Gemma 4 E4B IT Uncensored is a 8.0B-parameter open language model from TrevorJS in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Mixtral 8x7B v0.1

Mistral AI · 46.7B · runs from 19.8 GB

Mixtral-8x7B-v0.1 is Mistral AI's pretrained Sparse Mixture-of-Experts model, combining eight 7-billion-parameter experts for roughly 46.7 billion total parameters. As a base checkpoint, it has no built-in moderation or instruction-following behavior and is intended as a foundation for further fine-tuning rather than direct chat use. It supports a 32K token context window and is released under the Apache 2.0 license, allowing unrestricted commercial and research use. At around 47 billion total parameters, running it locally at 4-bit quantization needs roughly 24-32 GB of memory, putting it within reach of a single high-end consumer GPU or a dual-GPU setup.

Gemma 4 31B IT Uncensored

TrevorJS · 32.7B · runs from 15.5 GB

Gemma 4 31B IT Uncensored is a 32.7B-parameter open language model from TrevorJS in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Olmo 3.1 32B Instruct

Allen AI · 32.2B · runs from 9.7 GB

Olmo-3.1-32B-Instruct is Allen Institute for AI's 32-billion-parameter open instruction-tuned language model, part of the Olmo 3 family that also includes 7B and 32B "Think" reasoning variants. It is pretrained on the Dolma 3 corpus and post-trained in three stages on the Dolci datasets, covering math, code, chat, and instruction-following: supervised fine-tuning, direct preference optimization, and reinforcement learning from verifiable rewards. Ai2 reports it competitive with or ahead of similarly sized open models such as Qwen 3 32B and Gemma 3 27B on math, coding, and reasoning benchmarks, and it needs a multi-GPU workstation to run at full precision, considerably less once quantized. Context length is 65,536 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2025, a point release built on the same Olmo-3-32B base as the original Olmo 3 family.