All LLM Models

Browse 1475 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

MiMo v2 Flash

Xiaomi · 309.8B · runs from 85.6 GB

MiMo V2 Flash is Xiaomi's large-scale mixture-of-experts language model, built with nearly 310 billion total parameters. Designed for fast inference despite its size, the Flash variant prioritizes throughput and responsiveness, making it well-suited for interactive chat and real-time applications. Running it locally is a serious undertaking that demands high-end multi-GPU configurations, but it brings flagship-level Chinese and English language capabilities to users who have the hardware to support it.

Gemma 4 E4B IT Assistant

Google · 4B · runs from 2 GB

Gemma 4 E4B IT Assistant is a 4B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MiniCPM5 1B Agentic Tooluse Merged FP16

ewinregirgojr · 1.1B · runs from 0.8 GB

MiniCPM5 1B Agentic Tooluse Merged FP16 is a 1.1B-parameter open language model from ewinregirgojr in the MiniCPM family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Devstral 2 123B Instruct 2512

Mistral AI · 125.0B · runs from 35.4 GB

Devstral 2 123B Instruct 2512 is Mistral AI's large agentic coding model for software engineering tasks such as exploring codebases, editing multiple files, and running autonomous coding agents. Released as an FP8-quantized checkpoint of a roughly 125-billion-parameter dense model, it is a direct step up from the smaller Devstral Small line and pairs with Mistral's own Vibe CLI as well as third-party scaffolds like OpenHands and Claude Code. It scores 72.2% on SWE-Bench Verified, 61.3% on SWE-Bench Multilingual, and 32.6% on Terminal-Bench 2, competitive with or ahead of several much larger open models. At this size it needs a multi-GPU workstation even when quantized. Context length is 262,144 tokens. It is released under a Modified MIT License that blocks companies with over $20 million in monthly consolidated revenue from using it without a separate commercial license from Mistral AI. It was published in November 2025.

Qwen3 VL 2B Thinking

Alibaba · 2.1B · runs from 1.1 GB

Qwen3-VL-2B-Thinking is Alibaba's 2.1-billion-parameter vision-language model, the reasoning-enhanced "Thinking" edition of the smallest Qwen3-VL checkpoint, alongside a matching Instruct edition. Beyond image description and visual question answering, it acts as a visual agent capable of operating PC and mobile GUIs, generating code from screenshots or diagrams, and reasoning about 2D and 3D spatial relationships. Its OCR pipeline covers 32 languages and is tuned for low light, blur, and tilted text. At just over 2 billion parameters, it runs easily on a single modest consumer GPU, even unquantized. The model natively supports a 262,144 token context window, expandable to 1,010,000 tokens for long documents or hours of video. It is released under the Apache 2.0 license, and was published in October 2025. Compared with Qwen2.5-VL, Qwen3-VL adds DeepStack vision-feature fusion and timestamp-grounded video event localization.

Llama 3.2 11B Vision Instruct

Meta · 10.7B · runs from 5.0 GB

Llama 3.2 11B Vision Instruct is a smaller vision-language model in Meta's Llama 3 family, with roughly 11 billion parameters, built to process images together with text and follow chat-style instructions. It brings multimodal capability, such as visual question answering and image description, to a size that is far more approachable than the 90-billion-parameter variant from the same release. Released in September 2024 under Meta's Llama 3.2 community license, it does not carry a fully permissive open license but does allow broad use under Meta's terms. At around 11 billion parameters, it fits comfortably on a single mid-range to high-end consumer GPU once quantized to 4-bit, putting it within reach of enthusiast local setups.

Qwen3 30B A3B Abliterated

mlabonne · 30.5B · runs from 13.4 GB

Qwen3 30B A3B Abliterated is a 30.5B-parameter open language model from mlabonne in the Qwen 3 family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MiniMax M2.7 BF16 Ultra Uncensored Heretic

llmfan46 · 228.7B · runs from 97.8 GB

MiniMax M2.7 BF16 Ultra Uncensored Heretic is a 228.7B-parameter open language model from llmfan46 in the MiniMax family. It supports a context window of up to 204,800 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

OLMoE 1B 7B 0924 Instruct

Allen AI · 6.9B · runs from 3.5 GB

OLMoE 1B 7B 0924 Instruct is a 6.9B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Ornith 1.0 35B Uncensored Heretic

llmfan46 · 35.1B · runs from 15.3 GB

Ornith 1.0 35B Uncensored Heretic is a 35.1B-parameter open language model from llmfan46 in the Ornith family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MiniMax M2.7

MiniMax · 228.7B · runs from 63.5 GB

MiniMax-M2.7 is MiniMax's latest agentic Mixture-of-Experts model, continuing the M2 line with roughly 229 billion total parameters. It's built for complex coding and agentic workflows, including multi-agent "Agent Teams" collaboration, long-horizon software engineering tasks like debugging and refactoring, and general productivity work such as editing documents and spreadsheets, with part of its own development driven by an internal self-improvement loop. It supports a 204,800-token context window and is released under a custom license. At this scale, 4-bit quantization needs roughly 132GB of memory, so local deployment realistically requires a multi-GPU server rather than a single workstation.

Nanonets OCR S

nanonets · 3.8B · runs from 1.4 GB

Nanonets-OCR-s is Nanonets' image-to-markdown OCR model, built on top of Qwen2.5-VL-3B-Instruct, that goes beyond plain text extraction to produce structured markdown for downstream processing by other language models. It converts mathematical equations and formulas into LaTeX, describes embedded images and charts inside structured tags, isolates signatures and watermarks into their own tags, converts checkboxes into standard Unicode symbols, and extracts complex tables into both markdown and HTML formats. At under 4 billion parameters it is light enough to run on a single consumer GPU. Context length is 128,000 tokens. Nanonets has not published a license for the model on its Hugging Face model card. It was published in June 2025.

Granite 4.1 30B

IBM · 28.9B · runs from 8.8 GB

Granite 4.1 30B is IBM's 28.9-billion-parameter dense language model, tuned for chat, tool calling, and general assistant tasks as part of the Granite 4.1 family alongside smaller 3B and 8B variants. IBM built the 4.1 generation to be enterprise-friendly, with a focus on reliable tool calling and instruction following for agentic workflows. At this size, local inference calls for quantization and a capable GPU, fitting on a single high-end consumer or workstation card. The model supports a 131,072 token context window, enough for lengthy documents, codebases, or long multi-turn conversations. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in April 2026, it is the largest model in the Granite 4.1 generation, which succeeds Granite 4.0.

Gemma 4 E4B IT Uncensored

TrevorJS · 8.0B · runs from 3.9 GB

Gemma 4 E4B IT Uncensored is a 8.0B-parameter open language model from TrevorJS in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Mixtral 8x7B v0.1

Mistral AI · 46.7B · runs from 19.8 GB

Mixtral-8x7B-v0.1 is Mistral AI's pretrained Sparse Mixture-of-Experts model, combining eight 7-billion-parameter experts for roughly 46.7 billion total parameters. As a base checkpoint, it has no built-in moderation or instruction-following behavior and is intended as a foundation for further fine-tuning rather than direct chat use. It supports a 32K token context window and is released under the Apache 2.0 license, allowing unrestricted commercial and research use. At around 47 billion total parameters, running it locally at 4-bit quantization needs roughly 24-32 GB of memory, putting it within reach of a single high-end consumer GPU or a dual-GPU setup.

Gemma 4 31B IT Uncensored

TrevorJS · 32.7B · runs from 15.5 GB

Gemma 4 31B IT Uncensored is a 32.7B-parameter open language model from TrevorJS in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Olmo 3.1 32B Instruct

Allen AI · 32.2B · runs from 9.7 GB

Olmo-3.1-32B-Instruct is Allen Institute for AI's 32-billion-parameter open instruction-tuned language model, part of the Olmo 3 family that also includes 7B and 32B "Think" reasoning variants. It is pretrained on the Dolma 3 corpus and post-trained in three stages on the Dolci datasets, covering math, code, chat, and instruction-following: supervised fine-tuning, direct preference optimization, and reinforcement learning from verifiable rewards. Ai2 reports it competitive with or ahead of similarly sized open models such as Qwen 3 32B and Gemma 3 27B on math, coding, and reasoning benchmarks, and it needs a multi-GPU workstation to run at full precision, considerably less once quantized. Context length is 65,536 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2025, a point release built on the same Olmo-3-32B base as the original Olmo 3 family.

Phi 3.5 MoE Instruct

Microsoft · 41.9B · runs from 12.1 GB

Phi-3.5-MoE-instruct is Microsoft's Mixture-of-Experts model in the Phi-3.5 line, combining 16 experts of 3.8 billion parameters each, about 42 billion total, with only around 6.6 billion active per token. It is an instruction-tuned model refined with supervised fine-tuning, PPO, and DPO, built for multilingual reasoning tasks including code, math, and logic in memory- or latency-constrained deployments. It supports a 128K token context window and ships under the MIT license, allowing free commercial and research use. Because only about 6.6 billion parameters activate per token, it runs noticeably faster than its 42-billion-parameter size implies, though the full model still needs roughly 24 GB of memory at 4-bit quantization since all expert weights stay resident.

Granite 4.0 H Small

IBM · 32.2B · runs from 9.5 GB

Granite-4.0-H-Small is IBM's 32-billion-parameter instruction-tuned model in the Granite 4.0 line, fine-tuned from Granite-4.0-H-Small-Base for enterprise use: summarization, classification, extraction, question answering, retrieval-augmented generation, code tasks, function calling, and multilingual dialog. Architecturally it is a hybrid mixture-of-experts model with only 4 full-attention layers against 36 Mamba-2 state-space layers, and 72 experts with 10 active per token, giving about 9 billion active parameters out of 32 billion total; the Mamba-2 layers avoid growing a KV cache with context length, keeping long-context inference cheaper than a pure attention stack. It was trained with supervised fine-tuning, RL-based alignment, and model merging. At 32 billion parameters, it needs a high-end consumer GPU or multi-GPU setup once quantized. Context length is 131,072 tokens (128K). It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in October 2025.

HunyuanOCR

Tencent · 1.1B · runs from 0.7 GB

HunyuanOCR is Tencent's 1.1-billion-parameter vision-language model built for OCR and document understanding, unifying document parsing, text spotting, information extraction, and text-image translation in one model rather than a general chat assistant. The current checkpoint, HunyuanOCR-1.5, adds DFlash speculative decoding, where a lightweight draft model proposes tokens that the main model verifies in one pass, cutting latency on long structured outputs like tables and formulas. It also ships GGUF weights for llama.cpp, running on CPUs and laptop or consumer GPUs, not just server-grade vLLM. At just over 1 billion parameters, it is light enough for a single modest consumer GPU. Context length is 131,072 tokens; pretraining supports image resolutions up to 4K. It is released under the Tencent Hunyuan Community License, a custom license permitting commercial use outside the EU, UK, and South Korea and below 100 million monthly active users. It was published in November 2025.

Qwen3.8 27B OBLITERATED Mythos Class Agentic

medismera · 27.8B · runs from 12.6 GB

Qwen3.8 27B OBLITERATED Mythos Class Agentic is a 27.8B-parameter open language model from medismera in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen2 VL 7B Instruct

Alibaba · 8.3B · runs from 3.2 GB

Qwen2 VL 7B Instruct is Alibaba's 8.3-billion-parameter vision-language model from the original Qwen 2 generation, built to process images and video alongside text in a single conversation. It can describe images, answer visual questions, and reason over multi-image or video input, suiting it to multimodal chat and visual document tasks. At this parameter count, local inference is practical on a single mainstream or high-end consumer GPU once the weights are quantized, rather than requiring multi-GPU hardware. The model supports a 32K token context window, enough for moderate-length documents or multi-turn visual conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in August 2024, it was later superseded by Qwen2.5-VL-7B-Instruct, which its own model card lists as its successor.

SmolVLM 256M Instruct

Hugging Face · 256M · runs from 0.4 GB

SmolVLM-256M Instruct is Hugging Face's 256-million-parameter vision-language model, built on a compact SmolLM2-135M language backbone paired with a small 93-million-parameter SigLIP vision encoder. Hugging Face calls it the smallest publicly available multimodal model, designed for image captioning, visual question answering, and basic text transcription from images rather than open-ended chat. Its size makes it practical to run even on CPUs or entry-level GPUs, and on-device deployment is a stated use case. Context length is limited to 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in January 2025. Compared to the larger 2.2B SmolVLM2 sibling, it uses a smaller vision encoder and more aggressive image-token compression to further cut memory and latency, at some cost to accuracy.

Granite 4.0 Micro

IBM · 3.4B · runs from 1.4 GB

Granite-4.0-Micro is IBM's 3-billion-parameter long-context instruct model, fine-tuned from Granite-4.0-Micro-Base on a mix of permissively licensed open datasets and IBM's own synthetic data, using supervised fine-tuning, reinforcement-learning alignment, and model merging. Unlike some of its Granite 4.0 siblings (H Micro, H Tiny MoE, H Small MoE), which mix in Mamba2 layers or mixture-of-experts routing, Granite-4.0-Micro is a plain decoder-only dense transformer with grouped-query attention, RoPE, SwiGLU, and RMSNorm. It targets enterprise use cases such as summarization, RAG, tool-calling, and code completion, and at 3B parameters it runs comfortably on a single consumer GPU, even quantized on a laptop. Context length is 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in October 2025.

LFM2.5 1.2B Thinking

Liquid AI · 1.2B · runs from 0.9 GB

LFM2.5 1.2B Thinking is a 1.2B-parameter open language model from Liquid AI in the LFM2.5 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.8 2.4T A95B

Alibaba · 2446.2B · runs from 673.4 GB

Qwen3.8-2.4T-A95B is Alibaba's Qwen-Max-class Mixture-of-Experts model with 2.4 trillion total parameters and about 95 billion active per token, built on the Qwen3.5 architectural foundation with a hybrid layout mixing gated linear-attention and gated full-attention layers. It is a post-trained, agent-focused model for coding, professional work, research, and long-horizon multi-step tasks, with adjustable reasoning depth via a reasoning-effort setting. It natively supports a 262K token context window, extensible to roughly 1 million tokens, and is released under a custom Qwen license rather than a fully open one. At 2.4 trillion parameters, this is server and multi-GPU territory even at 4-bit; nearly everyone will use a hosted endpoint instead.

K2 Horizon 7B

IFM · 9.0B · runs from 4.4 GB

K2 Horizon 7B is a 9.0B-parameter open language model from IFM. It supports a context window of up to 524,288 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Mistral 7B v0.1

Mistral AI · 7.2B · runs from 3.6 GB

Mistral 7B v0.1 is the original base model from Mistral AI that helped reshape expectations for small open-weight language models when it launched in late 2023. As a pretrained foundation model without instruction tuning, it is designed for fine-tuning, research, and custom downstream tasks rather than direct conversational use. With 7 billion parameters and support for grouped-query attention and sliding-window attention, it remains a popular starting point for practitioners building specialized models. Its modest VRAM requirements of roughly 6 GB at 4-bit quantization keep it accessible on a wide range of consumer GPUs.

DeepSeek R1 Distill Qwen 14B Abliterated v2

huihui-ai · 14.8B · runs from 7.0 GB

DeepSeek R1 Distill Qwen 14B Abliterated v2 is a 14.8B-parameter open language model from huihui-ai in the DeepSeek R1 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.6 40B Claude 4.6 Opus Deckard Heretic Uncensored Thinking

DavidAU · 39.5B · runs from 80.0 GB

Qwen3.6 40B Claude 4.6 Opus Deckard Heretic Uncensored Thinking is a 39.5B-parameter open language model from DavidAU in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.