All LLM Models
Browse 880 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
NeoHorse 1 9B
TokenRhythm · 9.0B · runs from 4.4 GB
13.0K
1.0KNeoHorse 1 9B is a 9.0B-parameter open language model from TokenRhythm. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mistral Small 3.1 24B Instruct 2503
Mistral AI · 24.0B · runs from 7.3 GB
Mistral Small 3.1 24B Instruct 2503 is a 24-billion-parameter model from Mistral AI, the French AI lab, built on the earlier text-only Mistral Small 3 with added support for image input alongside text. It can reason about images in the same conversation as written prompts, useful for document understanding and multimodal chat. At 24 billion parameters, it needs quantization and a single high-end 24GB-class consumer or workstation GPU for local inference rather than budget hardware. The model supports a 128K token context window for long documents or extended conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in March 2025, it added vision understanding and a longer context window to the earlier text-only Mistral Small while keeping the same 24B parameter budget.
SmolVLM2 2.2B Instruct
Hugging Face · 2.2B · runs from 0.7 GB
SmolVLM2-2.2B Instruct is Hugging Face's 2.2-billion-parameter vision-language model, the largest member of the SmolVLM2 family, built to process interleaved text, images, and video within a single conversation. Beyond captioning and visual question answering, it is trained on additional video-instruction data for tasks like summarizing clips or answering questions about video content. At this size it still runs on a single consumer GPU, useful for on-device, resource-constrained deployment. The model supports a context window of 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2025 alongside smaller 500M and 256M SmolVLM2 siblings. Unlike those variants, which favor raw efficiency, the 2.2B model is tuned to be the most capable of the three on both image and video benchmarks.
SmolLM2 360M Instruct
Hugging Face · 362M · runs from 0.5 GB
SmolLM2 360M Instruct is an instruction-tuned model from Hugging Face that occupies the sweet spot between the 135M and 1.7B entries in the SmolLM2 lineup. At 360 million parameters, it offers noticeably better coherence and instruction-following ability than the smallest variants while still running comfortably on virtually any modern GPU or even on CPU. This model is well suited for on-device assistants, embedded applications, and rapid prototyping where you need real conversational ability without dedicating significant hardware resources. It handles short-form generation, summarization, and basic reasoning tasks with reasonable quality.
MiniCPM V 4.6
OpenBMB · 1.3B · runs from 0.9 GB
MiniCPM-V 4.6 is OpenBMB's 1.3-billion-parameter vision-language model, pairing a SigLIP2 image encoder with a small Qwen3.5-based backbone to read images, multi-image sequences, and video alongside text. It targets on-device and edge deployment above all else, with adapted builds for iOS, Android, and HarmonyOS, small enough to run on phones and any modern laptop without heavy hardware, quantized or not. The model offers a 262K token context window, unusually long for its size. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2026, introducing visual token compression that cuts visual-encoding computation by more than half versus earlier MiniCPM-V releases.
Cydonia 24B V4.3
TheDrummer · 23.6B · runs from 7.8 GB
Cydonia 24B V4.3 is a 23.6B-parameter open language model from TheDrummer. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 Omni 3B
Alibaba · 5.5B · runs from 1.7 GB
Qwen2.5-Omni-3B is Alibaba's 5.5-billion-parameter end-to-end omni-modal model, built to perceive text, images, audio, and video and respond with both text and natural streaming speech. It uses a Thinker-Talker architecture with a time-aligned position embedding (TMRoPE) that synchronizes video frames with audio, enabling real-time voice and video chat with chunked input and immediate output. As the smaller of the two Omni checkpoints, it runs on a single consumer GPU once quantized. Its language backbone supports a 32,768 token context window. It is released under Alibaba's Qwen Research License, which permits only non-commercial research and evaluation, unlike the Apache 2.0 used for Qwen's text-only models. Published in April 2025, about a month after the larger Qwen2.5-Omni-7B, it brings real-time multimodal chat to smaller-scale deployments.
Granite 4.0 H Tiny
IBM · 6.9B · runs from 2.4 GB
Granite 4.0 H Tiny is a 6.9B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
OlmOCR 2 7B 1025
Allen AI · 8.3B · runs from 2.7 GB
olmOCR-2-7B-1025 is Allen AI's 8.3-billion-parameter vision-language model, fine-tuned from Qwen2.5-VL-7B-Instruct specifically for document OCR rather than general chat. It converts scanned pages and PDFs into clean text, using GRPO reinforcement learning atop supervised fine-tuning to sharpen accuracy on math, tables, and tricky OCR cases. This is the full-precision BF16 release; Allen AI recommends its FP8 sibling for production use. At this size, it runs on a single mainstream-to-high-end consumer GPU once quantized. It supports a 128,000 token context window inherited from its Qwen2.5-VL base, useful for multi-page documents. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2025, it pairs with Allen AI's olmOCR toolkit, which handles page rendering and retries at scale via vLLM.
Deepseek Coder 6.7B Base
DeepSeek · 6.7B · runs from 3.2 GB
Deepseek Coder 6.7B Base is a 6.7B-parameter open language model from DeepSeek in the DeepSeek Coder family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
MiniCPM5 2B
OpenBMB · 2.5B · runs from 1.2 GB
MiniCPM5-2B is the second model in OpenBMB's MiniCPM5 series, a dense 2.5-billion-parameter causal language model built on a Llama-style architecture. This release is the final version, post-trained with reinforcement learning and on-policy distillation, and is aimed at local assistants, coding agents, tool-use workflows, and reasoning tasks where a small footprint matters, including on-device and edge deployment. It supports a 131,072-token context window, unusually long for its size, and ships under the Apache 2.0 license, allowing free commercial and non-commercial use. At roughly 2.5 billion parameters, it needs well under 2GB of memory at 4-bit quantization, so it runs comfortably on almost any modern GPU or laptop.
Medgemma 1.5 4B IT
Google · 4.3B · runs from 1.3 GB
MedGemma 1.5 4B is Google's 4.3-billion-parameter instruction-tuned vision-language model in the MedGemma line of health-AI foundation models, succeeding the original MedGemma 4B at the same size. It is built for developers creating healthcare applications, covering tasks such as medical image interpretation and clinical text understanding, and is not a validated diagnostic tool: outputs need independent verification and further evaluation before any clinical use. At this size it runs on a single consumer GPU once quantized. It is released under Google's Health AI Developer Foundations terms, a use-restricted license that requires accepting specific health-AI conditions on Hugging Face rather than a fully open one. Published in January 2026, it is the second 4B release in the MedGemma series.
NVIDIA Nemotron Nano 9B v2 Japanese
NVIDIA · 8.9B · runs from 4.4 GB
NVIDIA Nemotron Nano 9B v2 Japanese is a specialized variant of the Nemotron Nano 9B v2, fine-tuned for Japanese language understanding and generation. At 8.9 billion parameters, it maintains the same hardware-friendly footprint as the English version while delivering natural Japanese conversational ability. For users looking to run a Japanese-language assistant locally, this model offers a rare combination of compact size and dedicated language optimization from a major hardware vendor. It handles Japanese text with the fluency you'd expect from a purpose-built model rather than a multilingual afterthought.
Qwen3 VL 8B Thinking
Alibaba · 8.8B · runs from 3.0 GB
Qwen3 VL 8B Thinking is an 8.8-billion-parameter vision-language model from Alibaba's Qwen team, the reasoning-focused counterpart to the Qwen3-VL-8B Instruct model. It processes images, video, and text together and is tuned to work through problems step by step before answering, which tends to help on multi-step visual reasoning, STEM problems, and chart or document analysis. At under 9 billion parameters, it runs well on a single mainstream consumer GPU once quantized, making local multimodal use practical without server-class hardware. It supports a 262K token context window for long documents, multi-image input, or extended video. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2025 alongside the non-thinking Instruct variant, it trades some response speed for deeper visual-language reasoning.
Medgemma 4B IT
Google · 4.3B · runs from 1.3 GB
MedGemma 4B is Google's 4.3-billion-parameter instruction-tuned vision-language model, built on Gemma 3 and further trained for medical text and image understanding. It adds a medical image encoder covering chest X-rays, dermatology photos, histopathology slides, and fundus images, and can generate findings and answer questions about medical images. It is a foundation for developers building healthcare applications, not a validated diagnostic tool, and needs further evaluation before clinical use. It runs on a single consumer GPU once quantized. It inherits Gemma 3's 128,000 token context window. It is released under Google's Health AI Developer Foundations terms, a use-restricted license rather than a fully open one. Published in May 2025, it was Google's first instruction-tuned multimodal MedGemma release, alongside a larger text-only 27B variant.
Devstral Small 2507
Mistral AI · 23.6B · runs from 7.2 GB
Devstral Small 2507 is Mistral AI's agentic coding model, developed with All Hands AI and fine-tuned from the 24-billion-parameter Mistral Small 3.1 with its vision encoder removed to keep it text-only. It is built to explore codebases, edit multiple files, and drive software-engineering agents, using Mistral's function-calling format and a Tekken tokenizer with a 131K-token vocabulary. It supports a 128K token context window and is released under the Apache 2.0 license. At 24 billion parameters, Devstral is light enough to run on a single RTX 4090 or a Mac with around 32 GB of unified memory once quantized to 4-bit, making it practical for local coding-agent setups.
Phi 2
Microsoft · 2.8B · runs from 2.1 GB
Microsoft Phi 2 is a 2.8-billion parameter language model from Microsoft Research that pioneered the concept of small but highly capable language models. Released in late 2023, Phi 2 demonstrated that strategic data curation and training methodology could allow a sub-3B model to outperform many 7B and 13B models on reasoning and coding benchmarks. The model runs on virtually any modern GPU and even on CPU-only setups. While succeeded by Phi 3 and Phi 4, Phi 2 remains historically significant as the model that proved small-scale language models could be genuinely useful for practical tasks. Released under the MIT license.
LFM2 8B A1B
Liquid AI · 8.3B · runs from 2.7 GB
LFM2 8B A1B is Liquid AI's larger mixture-of-experts model, combining the company's novel hybrid architecture with approximately 8 billion total parameters. It uses a MoE design to keep active compute per token low while maintaining strong general performance across chat and reasoning tasks. For local users, it offers an intriguing alternative to conventional 8B transformers, with Liquid AI's architecture promising improved efficiency and throughput on consumer-grade hardware.
DeepSeek Coder v2 Lite Instruct
DeepSeek · 15.7B · runs from 7.2 GB
DeepSeek Coder V2 Lite Instruct is a code-focused mixture-of-experts model with 15.7 billion total parameters, trained to handle both programming tasks and general conversation. It supports a wide range of programming languages and excels at code generation, debugging, explanation, and refactoring. The MoE architecture keeps compute costs manageable despite the model's broad capabilities, and the Lite variant is sized to run on a single consumer GPU. For developers looking for a capable local coding assistant that can also handle general chat, this model offers an appealing combination of code specialization and practical hardware requirements.
SmolVLM2 500M Video Instruct
Hugging Face · 507M · runs from 0.5 GB
SmolVLM2-500M Video Instruct is Hugging Face's roughly 507-million-parameter vision-language model, a mid-sized member of the SmolVLM2 family purpose-built for analyzing video alongside images and text. It can answer questions about a clip, compare visual content across frames, or transcribe on-screen text, and Hugging Face designed it specifically for on-device video understanding where compute is limited. Its modest size makes it comfortable to run on a single consumer GPU. Context length is 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2025 alongside the larger 2.2B and smaller 256M SmolVLM2 models. Unlike the image-focused 256M model, both the 500M and 2.2B variants are explicitly trained and named for video understanding.
Jan Code 4B
janhq · 4.4B · runs from 2.4 GB
Jan Code 4B is a 4.4B-parameter open language model from janhq. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Unlimited OCR
Baidu · 3.3B · runs from 1.6 GB
Unlimited-OCR is Baidu's 3.3-billion-parameter vision-language model for OCR and document parsing, built to extend DeepSeek-OCR's approach further. It pairs a vision encoder with a Mixture-of-Experts decoder on DeepSeek's architecture, routing to 6 of 64 experts per token plus 2 shared experts, so only about 1.1 billion parameters activate per token even though every expert must still fit in memory. Its distinguishing idea, "one-shot long-horizon parsing," extends optical context compression to longer documents processed in a single pass. It is small enough to run on a single consumer GPU once quantized. Context length is 32,768 tokens, oriented around page and document parsing. It is released under the MIT license, a highly permissive option for commercial use. Published in June 2026, it has quickly picked up community support for vLLM inference and ms-swift fine-tuning.
Qwen2 0.5B Instruct
Alibaba · 494M · runs from 0.5 GB
Qwen2 0.5B Instruct is a 494M-parameter open language model from Alibaba in the Qwen 2 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 7B Instruct 1M
Alibaba · 7.6B · runs from 3.0 GB
Qwen2.5 7B Instruct 1M is a 7.6B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 1,010,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Hermes 4 14B
Nous Research · 14.8B · runs from 5.1 GB
Hermes 4 14B is a 14.8B-parameter open language model from Nous Research in the Hermes family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Magistral Small 2509
Mistral AI · 24.0B · runs from 7.3 GB
Magistral Small 2509 (also called Magistral Small 1.2) is Mistral AI's small reasoning model, built on Mistral Small 3.2 24B Instruct with added chain-of-thought reasoning trained through supervised fine-tuning on Magistral Medium traces followed by reinforcement learning. Unlike the text-only Magistral Small 1.1, this 1.2 release adds a vision encoder, so it can reason over images as well as text, wrapping its reasoning trace in dedicated [THINK]/[/THINK] tokens. It supports dozens of languages and is small enough to fit on a single consumer GPU once quantized. Context length is 131,072 tokens, though the card notes performance may degrade somewhat past 40,000 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2025.
Hermes 3 Llama 3.1 8B
Nous Research · 8.0B · runs from 3.3 GB
Hermes 3 Llama 3.1 8B is an 8-billion parameter instruction-tuned model by Nous Research, built on Meta's Llama 3.1 8B base. It is fine-tuned for advanced instruction following, multi-turn conversation, structured output, and creative roleplay scenarios. The Hermes series is known for producing highly steerable models that respond well to system prompts. This model supports a 128K token context window inherited from the Llama 3.1 architecture and runs efficiently on consumer GPUs with 8GB or more of VRAM. It is a popular choice among local inference enthusiasts who value strong instruction adherence and versatile conversational ability.
LFM2 VL 1.6B
Liquid AI · 1.6B · runs from 0.8 GB
LFM2-VL-1.6B is Liquid AI's 1.6-billion-parameter vision-language model, pairing a hybrid convolution-and-attention LFM2 language backbone (1.2B parameters) with a 400-million-parameter SigLIP2 NaFlex vision encoder. It targets image understanding and lightweight agentic tasks with variable-resolution inputs, tuned for low latency over heavy reasoning; Liquid AI recommends fine-tuning it for narrow use cases. It is small enough to run on a single modest consumer GPU. The model supports a 128,000 token context window. It is released under Liquid AI's LFM Open License v1.0, free including for commercial use under $10 million in annual revenue, above which a separate license applies. Published in August 2025 alongside a smaller 450M sibling, it claims roughly 2x faster GPU inference than comparable models while handling native 512x512 resolution without upscaling.
SmolLM2 1.7B Instruct
Hugging Face · 1.7B · runs from 1.4 GB
SmolLM2 1.7B Instruct is the largest instruction-tuned model in the SmolLM2 family, offering the best balance of capability and efficiency Hugging Face achieved with this generation. At 1.7 billion parameters it produces substantially more coherent and useful responses than its smaller siblings, handling multi-turn conversations, summarization, and simple reasoning tasks with competence. With VRAM requirements well under 4 GB at standard precision, this model runs effortlessly on entry-level GPUs, older laptops, and even some mobile devices. It is an excellent choice for developers building lightweight local assistants or chatbots who want genuine conversational quality without the hardware demands of larger models.
Qwen2.5 3B
Alibaba · 3.1B · runs from 1.6 GB
Qwen2.5 3B is a 3.1B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.