All LLM Models

Browse 1475 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

GLM 4.5 Air

Z.ai · 110.5B · runs from 30.8 GB

GLM-4.5-Air is Z.ai's more compact model in the GLM-4.5 series, a foundation model built for intelligent agents with 106 billion total parameters and about 12 billion active via its Mixture-of-Experts design. Like its larger sibling GLM-4.5, it offers both a thinking mode for complex reasoning and tool use and a non-thinking mode for quick replies, unifying reasoning, coding, and agentic capabilities in one checkpoint. It supports a 128K token context window and is released under the MIT license for commercial and research use. With around 106 billion total parameters, running it locally at 4-bit needs roughly 60 GB of memory, putting it in reach of a multi-GPU workstation or a high-memory Mac rather than a single card.

Medgemma 1.5 4B IT

Google · 4.3B · runs from 1.3 GB

MedGemma 1.5 4B is Google's 4.3-billion-parameter instruction-tuned vision-language model in the MedGemma line of health-AI foundation models, succeeding the original MedGemma 4B at the same size. It is built for developers creating healthcare applications, covering tasks such as medical image interpretation and clinical text understanding, and is not a validated diagnostic tool: outputs need independent verification and further evaluation before any clinical use. At this size it runs on a single consumer GPU once quantized. It is released under Google's Health AI Developer Foundations terms, a use-restricted license that requires accepting specific health-AI conditions on Hugging Face rather than a fully open one. Published in January 2026, it is the second 4B release in the MedGemma series.

Qwen3.8 27B DSpark

RadixArk · 27B · runs from 11.8 GB

Qwen3.8 27B DSpark is a 27B-parameter open language model from RadixArk in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Hemmingway 1

Altworld · 26.9B · runs from 8.1 GB

Hemmingway 1 is a 26.9B-parameter open language model from Altworld. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

NVIDIA Nemotron Nano 9B v2 Japanese

NVIDIA · 8.9B · runs from 4.4 GB

NVIDIA Nemotron Nano 9B v2 Japanese is a specialized variant of the Nemotron Nano 9B v2, fine-tuned for Japanese language understanding and generation. At 8.9 billion parameters, it maintains the same hardware-friendly footprint as the English version while delivering natural Japanese conversational ability. For users looking to run a Japanese-language assistant locally, this model offers a rare combination of compact size and dedicated language optimization from a major hardware vendor. It handles Japanese text with the fluency you'd expect from a purpose-built model rather than a multilingual afterthought.

Deepseek Coder 33B Instruct

DeepSeek · 33.3B · runs from 14.6 GB

DeepSeek-Coder-33B-Instruct is DeepSeek's 33.3-billion-parameter instruction-tuned code model, initialized from DeepSeek-Coder-33B-Base and further fine-tuned on 2 billion tokens of instruction data for chat-style code generation, completion, and fixing. The underlying Deepseek Coder family was pretrained from scratch on 2 trillion tokens (87% code, 13% natural language in English and Chinese) with project-level context and a fill-in-the-blank training objective, giving it strong project-level completion and infilling ability alongside state-of-the-art open-model results on HumanEval, MBPP, and related coding benchmarks at the time of release. At 33.3 billion parameters, it needs a high-end consumer GPU or a multi-GPU setup once quantized. Context length is 16,384 tokens. Model weights are released under DeepSeek's custom model license, which permits commercial use. It was published in November 2023, alongside 1.3B, 5.7B, and 6.7B siblings and the corresponding base checkpoint.

Qwen3 VL 8B Thinking

Alibaba · 8.8B · runs from 3.0 GB

Qwen3 VL 8B Thinking is an 8.8-billion-parameter vision-language model from Alibaba's Qwen team, the reasoning-focused counterpart to the Qwen3-VL-8B Instruct model. It processes images, video, and text together and is tuned to work through problems step by step before answering, which tends to help on multi-step visual reasoning, STEM problems, and chart or document analysis. At under 9 billion parameters, it runs well on a single mainstream consumer GPU once quantized, making local multimodal use practical without server-class hardware. It supports a 262K token context window for long documents, multi-image input, or extended video. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2025 alongside the non-thinking Instruct variant, it trades some response speed for deeper visual-language reasoning.

Medgemma 4B IT

Google · 4.3B · runs from 1.3 GB

MedGemma 4B is Google's 4.3-billion-parameter instruction-tuned vision-language model, built on Gemma 3 and further trained for medical text and image understanding. It adds a medical image encoder covering chest X-rays, dermatology photos, histopathology slides, and fundus images, and can generate findings and answer questions about medical images. It is a foundation for developers building healthcare applications, not a validated diagnostic tool, and needs further evaluation before clinical use. It runs on a single consumer GPU once quantized. It inherits Gemma 3's 128,000 token context window. It is released under Google's Health AI Developer Foundations terms, a use-restricted license rather than a fully open one. Published in May 2025, it was Google's first instruction-tuned multimodal MedGemma release, alongside a larger text-only 27B variant.

Devstral Small 2507

Mistral AI · 23.6B · runs from 7.2 GB

Devstral Small 2507 is Mistral AI's agentic coding model, developed with All Hands AI and fine-tuned from the 24-billion-parameter Mistral Small 3.1 with its vision encoder removed to keep it text-only. It is built to explore codebases, edit multiple files, and drive software-engineering agents, using Mistral's function-calling format and a Tekken tokenizer with a 131K-token vocabulary. It supports a 128K token context window and is released under the Apache 2.0 license. At 24 billion parameters, Devstral is light enough to run on a single RTX 4090 or a Mac with around 32 GB of unified memory once quantized to 4-bit, making it practical for local coding-agent setups.

Phi 2

Microsoft · 2.8B · runs from 2.1 GB

Microsoft Phi 2 is a 2.8-billion parameter language model from Microsoft Research that pioneered the concept of small but highly capable language models. Released in late 2023, Phi 2 demonstrated that strategic data curation and training methodology could allow a sub-3B model to outperform many 7B and 13B models on reasoning and coding benchmarks. The model runs on virtually any modern GPU and even on CPU-only setups. While succeeded by Phi 3 and Phi 4, Phi 2 remains historically significant as the model that proved small-scale language models could be genuinely useful for practical tasks. Released under the MIT license.

DeepSeek R1 Distill Qwen 32B Abliterated

huihui-ai · 32.8B · runs from 9.8 GB

DeepSeek R1 Distill Qwen 32B Abliterated is a 32.8B-parameter open language model from huihui-ai in the DeepSeek R1 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 4 26B A4B IT Uncensored Heretic

llmfan46 · 25.8B · runs from 11.6 GB

Gemma 4 26B A4B IT Uncensored Heretic is a 25.8B-parameter open language model from llmfan46 in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 4 26B A4B IT Assistant

Google · 26B · runs from 11.4 GB

Gemma 4 26B A4B IT Assistant is a 26B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen2.5 Coder 32B

Alibaba · 32.8B · runs from 9.8 GB

Qwen2.5-Coder-32B is Alibaba's 32.8-billion-parameter code-specialized base language model, the largest of six sizes in the Qwen2.5-Coder family (0.5B to 32B), continued-pretrained from Qwen2.5-32B on 5.5 trillion tokens of source code, text-code grounding data, and synthetic data. It is a raw pretrained model, not instruction-tuned, so it is meant as a foundation for building code assistants, agents, or chat models through further fine-tuning rather than for direct conversational use. Alibaba states its coding ability rivals GPT-4o, and at this size it needs a high-end consumer GPU or multi-GPU setup once quantized, more at full precision. Context length defaults to 32,768 tokens, extendable to 131,072 tokens with YaRN scaling. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in November 2024.

MiniMax M2.1

MiniMax · 228.7B · runs from 63.5 GB

MiniMax M2.1 is an earlier generation of MiniMax's large mixture-of-experts model series, featuring the same 228 billion total parameter architecture as its successor. It offers strong multilingual performance across Chinese and English tasks, including conversation, reasoning, and content generation. While M2.5 refines the formula, M2.1 remains a capable option for users with the multi-GPU hardware needed to host a model of this scale locally.

LFM2 8B A1B

Liquid AI · 8.3B · runs from 2.7 GB

LFM2 8B A1B is Liquid AI's larger mixture-of-experts model, combining the company's novel hybrid architecture with approximately 8 billion total parameters. It uses a MoE design to keep active compute per token low while maintaining strong general performance across chat and reasoning tasks. For local users, it offers an intriguing alternative to conventional 8B transformers, with Liquid AI's architecture promising improved efficiency and throughput on consumer-grade hardware.

DeepSeek Coder v2 Lite Instruct

DeepSeek · 15.7B · runs from 7.2 GB

DeepSeek Coder V2 Lite Instruct is a code-focused mixture-of-experts model with 15.7 billion total parameters, trained to handle both programming tasks and general conversation. It supports a wide range of programming languages and excels at code generation, debugging, explanation, and refactoring. The MoE architecture keeps compute costs manageable despite the model's broad capabilities, and the Lite variant is sized to run on a single consumer GPU. For developers looking for a capable local coding assistant that can also handle general chat, this model offers an appealing combination of code specialization and practical hardware requirements.

Qwen3 Omni 30B A3B Thinking

Alibaba · 31.7B · runs from 14.8 GB

Qwen3-Omni-30B-A3B-Thinking is the reasoning-focused variant of Alibaba's Qwen3-Omni family, a mixture-of-experts model that accepts text, audio, image, and video input and reasons over it with explicit chain-of-thought before answering in text. Unlike the Instruct variant, it keeps only the "thinker" component and drops the speech-generating "talker" and audio codec, so it outputs text only, not speech. Its backbone still routes across 128 experts with 8 active per token, roughly 3 billion active out of about 31.7 billion total, and needs a high-end consumer GPU or multi-GPU setup once quantized. Context length is 65,536 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2025, a few days ahead of the Instruct variant that adds real-time speech generation on top of the same reasoning core.

SmolVLM2 500M Video Instruct

Hugging Face · 507M · runs from 0.5 GB

SmolVLM2-500M Video Instruct is Hugging Face's roughly 507-million-parameter vision-language model, a mid-sized member of the SmolVLM2 family purpose-built for analyzing video alongside images and text. It can answer questions about a clip, compare visual content across frames, or transcribe on-screen text, and Hugging Face designed it specifically for on-device video understanding where compute is limited. Its modest size makes it comfortable to run on a single consumer GPU. Context length is 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2025 alongside the larger 2.2B and smaller 256M SmolVLM2 models. Unlike the image-focused 256M model, both the 500M and 2.2B variants are explicitly trained and named for video understanding.

Jan Code 4B

janhq · 4.4B · runs from 2.4 GB

Jan Code 4B is a 4.4B-parameter open language model from janhq. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatFunctionsCode

Unlimited OCR

Baidu · 3.3B · runs from 1.6 GB

Unlimited-OCR is Baidu's 3.3-billion-parameter vision-language model for OCR and document parsing, built to extend DeepSeek-OCR's approach further. It pairs a vision encoder with a Mixture-of-Experts decoder on DeepSeek's architecture, routing to 6 of 64 experts per token plus 2 shared experts, so only about 1.1 billion parameters activate per token even though every expert must still fit in memory. Its distinguishing idea, "one-shot long-horizon parsing," extends optical context compression to longer documents processed in a single pass. It is small enough to run on a single consumer GPU once quantized. Context length is 32,768 tokens, oriented around page and document parsing. It is released under the MIT license, a highly permissive option for commercial use. Published in June 2026, it has quickly picked up community support for vLLM inference and ms-swift fine-tuning.

Qwen2 0.5B Instruct

Alibaba · 494M · runs from 0.5 GB

Qwen2 0.5B Instruct is a 494M-parameter open language model from Alibaba in the Qwen 2 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen2.5 7B Instruct 1M

Alibaba · 7.6B · runs from 3.0 GB

Qwen2.5 7B Instruct 1M is a 7.6B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 1,010,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Hermes 4 14B

Nous Research · 14.8B · runs from 5.1 GB

Hermes 4 14B is a 14.8B-parameter open language model from Nous Research in the Hermes family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatReasoningRoleplay

Magistral Small 2509

Mistral AI · 24.0B · runs from 7.3 GB

Magistral Small 2509 (also called Magistral Small 1.2) is Mistral AI's small reasoning model, built on Mistral Small 3.2 24B Instruct with added chain-of-thought reasoning trained through supervised fine-tuning on Magistral Medium traces followed by reinforcement learning. Unlike the text-only Magistral Small 1.1, this 1.2 release adds a vision encoder, so it can reason over images as well as text, wrapping its reasoning trace in dedicated [THINK]/[/THINK] tokens. It supports dozens of languages and is small enough to fit on a single consumer GPU once quantized. Context length is 131,072 tokens, though the card notes performance may degrade somewhat past 40,000 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2025.

Gemma 2 27B IT

Google · 27.2B · runs from 9.0 GB

Google Gemma 2 27B IT is a 27.2-billion parameter instruction-tuned model from Google's Gemma 2 generation. It is a text-only chat model optimized for conversational use, reasoning, and instruction following. Gemma 2 27B IT was one of the strongest openly available models in its size class at release. The model requires a GPU with at least 24GB of VRAM for quantized local inference. It is widely supported by popular inference engines and remains a strong choice for users seeking high-quality local chat without needing 70B-class hardware. Released under the Gemma license.

Hermes 3 Llama 3.1 8B

Nous Research · 8.0B · runs from 3.3 GB

Hermes 3 Llama 3.1 8B is an 8-billion parameter instruction-tuned model by Nous Research, built on Meta's Llama 3.1 8B base. It is fine-tuned for advanced instruction following, multi-turn conversation, structured output, and creative roleplay scenarios. The Hermes series is known for producing highly steerable models that respond well to system prompts. This model supports a 128K token context window inherited from the Llama 3.1 architecture and runs efficiently on consumer GPUs with 8GB or more of VRAM. It is a popular choice among local inference enthusiasts who value strong instruction adherence and versatile conversational ability.

LFM2 VL 1.6B

Liquid AI · 1.6B · runs from 0.8 GB

LFM2-VL-1.6B is Liquid AI's 1.6-billion-parameter vision-language model, pairing a hybrid convolution-and-attention LFM2 language backbone (1.2B parameters) with a 400-million-parameter SigLIP2 NaFlex vision encoder. It targets image understanding and lightweight agentic tasks with variable-resolution inputs, tuned for low latency over heavy reasoning; Liquid AI recommends fine-tuning it for narrow use cases. It is small enough to run on a single modest consumer GPU. The model supports a 128,000 token context window. It is released under Liquid AI's LFM Open License v1.0, free including for commercial use under $10 million in annual revenue, above which a separate license applies. Published in August 2025 alongside a smaller 450M sibling, it claims roughly 2x faster GPU inference than comparable models while handling native 512x512 resolution without upscaling.

SmolLM2 1.7B Instruct

Hugging Face · 1.7B · runs from 1.4 GB

SmolLM2 1.7B Instruct is the largest instruction-tuned model in the SmolLM2 family, offering the best balance of capability and efficiency Hugging Face achieved with this generation. At 1.7 billion parameters it produces substantially more coherent and useful responses than its smaller siblings, handling multi-turn conversations, summarization, and simple reasoning tasks with competence. With VRAM requirements well under 4 GB at standard precision, this model runs effortlessly on entry-level GPUs, older laptops, and even some mobile devices. It is an excellent choice for developers building lightweight local assistants or chatbots who want genuine conversational quality without the hardware demands of larger models.

Qwen2.5 3B

Alibaba · 3.1B · runs from 1.6 GB

Qwen2.5 3B is a 3.1B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.