All LLM Models
Browse 1699 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Model List
DeepSeek R1 0528 Qwen3 8B
DeepSeek · 8.2B · runs from 7.6 GB at Q4
349.9K
1.1KWith about 8 billion parameters, DeepSeek R1 0528 Qwen3 8B is a dated May 2025 release in DeepSeek's R1 line, built on a Qwen3-based architecture rather than DeepSeek's own base model. It supports explicit step-by-step reasoning, generating intermediate reasoning traces before producing a final answer, which sets it apart from standard instruction-tuned chat models that answer directly. Its modest size means it runs comfortably on modest consumer GPUs, making it accessible for local experimentation with reasoning-style outputs without specialized hardware. It offers a 128K token context window, ample for most reasoning and coding tasks, and is released under the MIT license, allowing broad local and commercial use.
Qwen2.5 0.5B Instruct
Alibaba · 494M · runs from 0.8 GB at Q4
Qwen2.5 0.5B Instruct is the smallest instruction-tuned model in Alibaba Cloud's Qwen 2.5 family, with just 494 million parameters. It is designed for ultra-lightweight deployment scenarios where minimal hardware resources are available, running comfortably on virtually any modern GPU or even CPU-only configurations. Despite its tiny footprint, the model supports a 128K token context window and can handle basic chat, simple summarization, and lightweight instruction following. It is primarily useful for edge deployment, experimentation, and prototyping where model size is a critical constraint. Released under the Apache 2.0 license.
MiniCPM SALA
OpenBMB · 9.5B · runs from 8.9 GB at Q4
MiniCPM SALA is a 9.5B-parameter open language model from OpenBMB in the MiniCPM family. It supports a context window of up to 524,288 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.5 2B
Alibaba · 2.3B · runs from 1.9 GB at Q4
Qwen3.5 2B is a 2.3-billion-parameter model from Alibaba's Qwen team, one of the smaller entries in the Qwen3.5 lineup (0.8B–9B) built to handle text and image input together. It uses a hybrid architecture mixing linear-attention layers with periodic full-attention layers to keep long-context inference efficient. As a vision-capable model it can read and reason about images alongside written prompts, and its small size lets it run on a laptop or even a phone once quantized, without a dedicated GPU. It supports an unusually long 262K token context window for its size. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and was published in late February 2026 as part of Alibaba's push toward compact, natively multimodal edge models.
MiniCPM V 4.7 35B A3B
OpenBMB · 35.2B · runs from 22.1 GB at Q4
MiniCPM-V 4.7 35B-A3B is OpenBMB's vision-language model and the first mixture-of-experts entry in the MiniCPM-V series, with about 35 billion total parameters and roughly 4 billion active per token. It reads single images, multiple images, and video, and is built with phones and other edge devices in mind: the card describes a scheduling technique and layer-wise quantization that let it run on an iPhone 17 Pro Max at over 13 tokens per second. On a PC, a Q4 build fits a 24 GB GPU or a 32 GB unified-memory machine, and the small active set keeps decoding fast. The context window is 262,144 tokens. It is released under the Apache 2.0 license, so commercial use is unrestricted. The vision encoder compresses visual tokens early and supports mixed 4x and 16x compression, letting you trade image detail for speed. It works with llama.cpp, Ollama, vLLM, and SGLang, and GGUF, AWQ, GPTQ, and FP8 variants are available.
GLM 5.3
Z.ai · 753.3B · runs from 453.8 GB at Q4
GLM 5.3 continues Z.ai's GLM 5 line as a mixture-of-experts model with roughly 753 billion total parameters. Like other MoE designs, it activates only a fraction of those parameters for each token, which keeps inference more efficient relative to a dense model of comparable total size. The model supports a 1M token context window for very long documents or sessions, and is distributed under a custom license rather than the MIT terms used elsewhere in the family. Given its total parameter count, GLM 5.3 needs multi-GPU or server-class hardware to run; most users will access it through a hosted endpoint instead of locally.
Spark X2.6 4B
XHToken · 4.1B · runs from 4.3 GB at Q4
Spark-X2.6-4B is a compact 4.1-billion-parameter dense language model from XHToken, aimed at on-device agents and everyday real-world tasks. It is the post-trained version of Spark-X2.6-4B-Base and keeps exactly the same architecture as Spark-X2.5-4B, with XHToken crediting improved post-training for gains in coding, agentic tool use, long-context handling, multilingual work and reasoning. It supports English and Chinese and has three configurable thinking levels (none, low and high), so an application can trade reasoning quality against latency. XHToken says it matches or exceeds some 9B-class models on several benchmarks. Context length is 1,048,576 tokens, though XHToken notes that using the full window needs plenty of device memory and recommends lowering the limit when necessary. It is released under the Apache 2.0 license, permitting commercial use, and was published in October 2026. At 4.1 billion parameters it is small enough for a laptop, a mid-range consumer GPU or a recent phone-class device once quantized, and keeping the context window modest keeps memory use low.
Unlimited OCR
Baidu · 3.3B · runs from 3.3 GB at Q4
Unlimited-OCR is Baidu's 3.3-billion-parameter vision-language model for OCR and document parsing, built to extend DeepSeek-OCR's approach further. It pairs a vision encoder with a Mixture-of-Experts decoder on DeepSeek's architecture, routing to 6 of 64 experts per token plus 2 shared experts, so only about 1.1 billion parameters activate per token even though every expert must still fit in memory. Its distinguishing idea, "one-shot long-horizon parsing," extends optical context compression to longer documents processed in a single pass. It is small enough to run on a single consumer GPU once quantized. Context length is 32,768 tokens, oriented around page and document parsing. It is released under the MIT license, a highly permissive option for commercial use. Published in June 2026, it has quickly picked up community support for vLLM inference and ms-swift fine-tuning.
Ornith 1.5 122B
ornith-ai · 125.1B · runs from 75.5 GB at Q4
Ornith-1.5-122B is a mixture-of-experts language model from ornith-ai, the largest-capability member of the Ornith-1.5 family. It extends Ornith-1.0, which was built on Qwen3.5 and Gemma 4, and the team describes it as the product of an end-to-end self-improvement loop covering task generation, scaffolds, and solution rollouts. It has about 125 billion total parameters with roughly 12.7 billion active per token. A Q4 build needs roughly 71 GB of memory, and the card says the model serves on a single 80 GB GPU. The context window is 262,144 tokens, and the card describes extending it to roughly 1 million with YaRN scaling. It is released under the MIT license, which permits commercial use, and was published on October 16, 2026. The card reports results on Terminal-Bench 2.1, SWE-Bench Verified and Pro, MCP-Atlas, and Toolathlon-Verified. It gives vLLM and SGLang serving recipes with tool-call and reasoning parsers, and a quantized Mobile variant exists for edge devices.
Flint
RecursiveSI · 39.8B · runs from 25.0 GB at Q4
Flint is a mixture-of-experts language model for coding agents from Recursive. It is built to investigate codebases, fix bugs, implement features, and write tests, with reasoning and multi-turn tool use over text input and output. The card gives 39.8 billion total parameters, about 2.5 billion active per token, 256 routed experts with 6 selected per token, and a mix of sliding-window and full-attention layers. A Q4 build needs roughly 23 GB of memory, so it fits a 24 GB GPU only tightly and suits a 32 GB card or a unified-memory machine better. The documented context configuration is 262,144 tokens, shared between prompt and output. It is released under the Apache 2.0 license, which permits commercial use, and was published on October 21, 2026. Its card reports 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2, and 74.2% on SWE-bench Verified. The BF16 checkpoint is about 80 GB, and the card describes serving it with SGLang on one B300 or H200 GPU.
Sage A1.5
InternScience · 36.0B · runs from 22.0 GB at Q4
Sage-A1.5 is InternScience's agentic mixture-of-experts model for long-horizon scientific problem solving, built on Qwen3.6 and the earlier Agents-A1 from Shanghai AI Laboratory. It has about 36 billion parameters in total with roughly 4.7 billion active per token, on the Qwen3.5-MoE vision-language architecture, so it can take images as well as text. InternScience highlights three abilities: orchestrating parallel subagents with isolated contexts, an opt-in verification mode where the model reviews its own progress mid-reasoning, and open-ended scientific exploration, alongside native function calling and tool use. All experts must still fit in memory even though only a fraction runs per token. Context length is 262,144 tokens natively, and InternScience describes serving it at 512K with a rope-scaling setting. It is released under the Apache 2.0 license, permitting commercial use, and was published in October 2026. Only about 4.7 billion parameters run per token, so generation is fast for its size once the weights are loaded; a 4-bit quantization is the practical route onto a single high-memory GPU or a unified-memory machine.
Qwen2.5 VL 32B Instruct
Alibaba · 33.5B · runs from 24.7 GB at Q4
Qwen2.5 VL 32B Instruct is a 33.5-billion-parameter vision-language model from Alibaba's Qwen team, able to process images and text together for document parsing, chart reading, and visual question answering. It was tuned with reinforcement learning on top of the original Qwen2.5-VL release for more detailed, better-formatted answers and sharper accuracy on math and visual-logic problems. At this size, local inference needs quantization and a single high-end 24-32GB-class consumer or workstation GPU rather than lower-end hardware. The model supports a 128K token context window, enough for lengthy documents or multi-image inputs. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and was published in March 2025 as a mid-sized addition to the Qwen2.5-VL lineup, between the smaller 7B model and the flagship 72B version.
LFM2.5 1.2B Instruct
Liquid AI · 1.2B · runs from 1.2 GB at Q4
LFM2.5 1.2B Instruct is an instruction-tuned model from Liquid AI that uses a novel hybrid architecture combining state-space models with attention mechanisms. At just 1.2 billion parameters, it is exceptionally lightweight and can run on virtually any hardware, including laptops and edge devices. Liquid AI's unconventional architecture aims to deliver better efficiency and longer context handling than traditional transformer models at this scale, making it an interesting option for users exploring alternatives to standard transformer-based LLMs.
Mellum2.1 12B A2.5B Thinking
JetBrains · 12.1B · runs from 7.8 GB at Q4
Mellum2.1 Thinking is JetBrains' mixture-of-experts reasoning model featuring 12 billion total parameters and 2.5 billion active ones, designed for agentic tasks within code repositories alongside challenging coding, math, and logic problems. It contains 28 layers with 64 experts per layer, activating 8 per token, and it was post-trained from Mellum2-12B-A2.5B-Base primarily using reinforcement learning, including sessions inside real repositories employing shell commands and file-editing utilities. The card lists self-evaluated scores of 82.0 on LiveCodeBench v6, 47.0 on SWE-bench Verified, and 83.3 on AIME 25/26. With only 2.5 billion parameters active at any given time, it should decode rapidly; furthermore, quantized to 4-bit, the model fits on a single consumer GPU or laptop provided there is sufficient memory for the full 12 billion weights. The context window reaches 131,072 tokens, with sliding-window attention applied to three of every four layers. Released under the Apache 2.0 license, it allows unrestricted commercial and research usage. Published in September 2026, this model succeeds Mellum2 Thinking, retaining the same architecture while featuring an enhanced post-training phase.
Qwen3.5 0.8B
Alibaba · 873M · runs from 0.9 GB at Q4
Qwen3.5-0.8B is the smallest model in Alibaba's Qwen3.5 generation, a dense sub-billion-parameter vision-language model using the same hybrid Gated DeltaNet and gated-attention design as the rest of the family. It is a post-trained, instruction-tuned release, and its publisher notes that at this scale it is best suited to prototyping, task-specific fine-tuning, and other research or development purposes rather than production chat. It supports a 262,144-token native context window and is released under the Apache 2.0 license. At under one billion parameters, it needs well under a gigabyte of memory at 4-bit, so it runs on essentially any modern laptop or GPU.
Inkling Small
Thinking Machines · 266.0B · runs from 162.7 GB at Q4
Inkling Small is a vision-language chat model from Thinking Machines, positioned as the smaller of two models in the Inkling family, though it still totals roughly 266 billion parameters. It accepts image input alongside text and is designed for general conversational use, combining visual understanding with standard dialogue capability. It is released under the Apache 2.0 license, permitting free local and commercial use. At this parameter scale, Inkling Small needs multi-GPU or server-class hardware to run at all; most users will reach it through a hosted endpoint rather than running it locally on their own systems.
Apodex 1.1 Mini
apodex · 36.0B · runs from 22.0 GB at Q4
Apodex 1.1 Mini is a 36.0B-parameter open language model from apodex. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron 3.5 Bolt E4B BF16
NVIDIA · 8.0B · runs from 5.6 GB at Q4
NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 is a small language model from NVIDIA, post-trained from Google's Gemma-4-E4B-it and released in full BF16 precision. It targets local voice and device assistants, interactive personas, and embedded interfaces, with better multi-turn tool use, instruction following, and input grounding than the base model. It has about 8 billion parameters, 4 billion of them effective. A Q4 build needs roughly 5 GB of memory, and NVIDIA also publishes a separate 4 GB Q4_K_M release, so it runs on a modest GPU or a laptop. The context length is up to 64K tokens for input and output each. It is released under the OpenMDW-1.1 license, and the card says it is ready for commercial or non-commercial use. It was published on October 5, 2026. Its card reports 63.79 on IFBench against 42.84 for Gemma 4 E4B-it, and 68.51 on BFCL-v3. Thinking can be switched on or off in the chat template, and vLLM is the documented serving path.
Qwen3 VL 30B A3B Instruct
Alibaba · 31.1B · runs from 19.7 GB at Q4
Qwen3 VL 30B A3B Instruct is Alibaba's 31-billion-parameter mixture-of-experts vision-language model in the Qwen 3 lineup, with about 3 billion parameters active per token (the A3B in its name). Because only the active experts run for each token, inference is faster than a similarly sized dense model, while all the weights still need to fit in memory. It handles images and video alongside text, supporting visual question answering, document and chart reading, and multi-image reasoning. At this size, local inference calls for quantization and a single high-end consumer GPU. The model supports a 262K token context window, suited to long documents, video transcripts, or extended visual conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and was published in September 2025 as part of Qwen's third-generation vision-language family.
Ornith 1.0 9B
Deep Reinforce · 9.4B · runs from 6.5 GB at Q4
Ornith 1.0 9B is Deep Reinforce's smaller, dense sibling to Ornith 1.0 35B, with roughly 9.4 billion parameters and no mixture-of-experts routing. It is tuned for general chat and instruction-following use. Models in this parameter range run comfortably on modest consumer GPUs, and remain usable on lower-VRAM setups at higher quantization, making it a practical option for local deployment without high-end hardware. The model supports a 256K token context window, matching its larger sibling despite the smaller parameter count. It is released under the MIT license and was published on June 21, 2026, giving Deep Reinforce two size options in the Ornith 1.0 line for different hardware budgets.
GPT OSS 120B
OpenAI · 116.8B · runs from 70.8 GB at Q4
GPT-OSS 120B is the larger of OpenAI's open-source model releases, bringing 120.4 billion parameters of GPT-lineage capability to the open-weight ecosystem. It represents near-frontier performance across reasoning, knowledge, code generation, and conversational tasks, rivaling top proprietary offerings in many benchmarks. Running this model locally is a serious hardware commitment, typically requiring multiple high-VRAM GPUs or a professional-grade setup with 80+ GB of combined VRAM even at aggressive quantization levels. It is best suited for enthusiasts with multi-GPU rigs or workstation hardware who want the strongest possible local model from OpenAI's catalog.
Gemma 3 4B IT
Google · 4.3B · runs from 5 GB at Q4
Gemma 3 4B IT is a small, roughly 4.3-billion-parameter model from Google's earlier Gemma 3 generation, tuned for instruction-following and chat. It accepts image input alongside text, so it can handle visual question answering and image-grounded prompts in addition to text-only conversation. At this size, the model is comfortable on modest consumer hardware, running even on integrated graphics at low quantization, making it one of the most accessible vision-capable models for local use. Released on February 20, 2025, Gemma 3 4B IT predates the Gemma 4 family and is distributed under Google's Gemma license terms rather than a standard open-source license. Its compact size and image-input support make it a practical choice for lightweight, on-device multimodal applications.
Mistral 7B v0.3
Mistral AI · 7.2B · runs from 6.8 GB at Q4
Mistral 7B v0.3 is a 7.2B-parameter open language model from Mistral AI in the Mistral family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Laguna S 2.1
poolside · 117.6B · runs from 72.5 GB at Q4
Laguna S 2.1 is a 117.6B-parameter open language model from poolside. It supports a context window of up to 1,048,576 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 26B A4B IT Qat Q4 0 Unquantized
Google · 25.8B · runs from 16.5 GB at Q4
Gemma 4 26B A4B IT Qat Q4 0 Unquantized is a 25.8B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Hy MT2 1.8B
Tencent · 2.0B · runs from 2.6 GB at Q4
Hy MT2 1.8B is a 2-billion-parameter translation model from Tencent's Hunyuan team, part of a three-size Hy-MT2 family (1.8B, 7B, and a 30B mixture-of-experts variant) built specifically for machine translation rather than general chat. It supports translation across roughly three dozen languages and follows translation-specific instructions, such as adjusting tone or terminology. It runs on modest consumer hardware, including laptops, and Tencent has also released extreme-quantized versions shrunk to a few hundred megabytes for on-device use. It supports a 262K token context window, letting it translate long documents in a single pass. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in May 2026, it is the smallest member of a translation-focused family Tencent positions as a self-hostable alternative to commercial translation APIs.
Qwen3 VL 32B Instruct
Alibaba · 33.4B · runs from 23 GB at Q4
Qwen3 VL 32B Instruct is Alibaba's largest dense model in the initial Qwen3-VL lineup, a 33.4-billion-parameter vision-language model built to process images and text in one pass. It handles image description, visual question answering, and document understanding, and its visual-agent tuning lets it read GUI screenshots and reason about on-screen elements. Local inference calls for quantization and a capable GPU, fitting on a single high-end consumer or workstation card. The model supports a 262,144 token context window, enough for long documents or extended multi-turn conversations. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2025 alongside the 2B and 8B models, it shares the family's long native context and video-understanding capabilities, giving stronger multimodal reasoning than the smaller variants.
Qwen3 30B A3B
Alibaba · 30.5B · runs from 19.4 GB at Q4
Qwen3 30B A3B is a Mixture of Experts (MoE) model from Alibaba Cloud's Qwen 3 series, with 30 billion total parameters and approximately 3 billion active parameters per forward pass. The MoE architecture delivers quality significantly above what a standard 3B dense model could achieve, while keeping per-token compute costs low. It supports hybrid thinking mode for flexible reasoning. The model requires VRAM proportional to its full 30B parameter count for weight loading, but its low active parameter count results in fast inference throughput. It is an efficient option for users who want quality beyond dense small models without the full cost of larger architectures. Released under the Apache 2.0 license.
Qwen2.5 72B Instruct
Alibaba · 72.7B · runs from 49.3 GB at Q4
Qwen2.5 72B Instruct is the flagship model of the Qwen 2.5 series from Alibaba Cloud, with 72.7 billion parameters. It is instruction-tuned for conversational use and excels across reasoning, coding, mathematics, and multilingual tasks. Qwen2.5 72B delivers performance competitive with leading open-weight 70B-class models while supporting a 128K token context window and structured output generation. The model uses a Transformer architecture with grouped-query attention and was pretrained on a diverse multilingual corpus of over 18 trillion tokens. Running it locally requires high-VRAM GPUs or multi-GPU setups, though quantized formats make it accessible on workstation-class hardware. Released under the Apache 2.0 license.
Qwen3 VL 2B Instruct
Alibaba · 2.1B · runs from 3.5 GB at Q4
Qwen3 VL 2B Instruct is Alibaba's smallest model in the Qwen3-VL lineup, a 2.1-billion-parameter vision-language model built to handle images and text together. It performs image captioning, visual question answering, and document reading, and its visual-agent tuning lets it interpret GUI screenshots for simple automation. Its small size suits on-device and edge deployment, running comfortably on modest consumer GPUs or even some laptops once quantized. The model supports a 262,144 token context window, enough for lengthy documents or multi-turn conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in October 2025 alongside the 8B and 32B models, it targets edge and mobile use cases, trading some visual reasoning depth for a much smaller footprint.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.