All LLM Models
Browse 1323 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Model List
Qwen2.5 Coder 32B Instruct
Alibaba · 32.8B · runs from 24.3 GB at Q4
546.0K
2.2KQwen2.5 Coder 32B Instruct is a 32.8-billion parameter code-specialized model from Alibaba Cloud, instruction-tuned for programming assistance and code generation. It is trained on a large corpus of source code alongside natural language data, making it highly capable for tasks such as code completion, debugging, code explanation, and software engineering dialogue. The model supports a 128K token context window and delivers code generation quality competitive with the best open-weight coding models at any scale. It requires a GPU with at least 24GB of VRAM for quantized inference. Released under the Apache 2.0 license.
Muse Glimmer 30B
Meta · 29.8B · runs from 19.6 GB at Q4
Muse Glimmer 30B is Meta's 30-billion-parameter dense vision-language model, purpose-built for local, always-on AI agents rather than general chat. It pairs a text decoder with a dedicated image encoder for reasoning over screenshots, charts, and documents, and includes native tool-calling with a separate reasoning channel so it can plan multi-step actions and recover from failures. At this size, local inference calls for quantization and a capable GPU; it fits a single high-end 24-32GB-class card, matching Meta's goal of running it entirely on consumer machines. The model supports a 131K token context window, enough for extended agent sessions. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in August 2026, it is Meta's first open-weight model since Llama 4, and ships without the Llama licenses' monthly-active-user cap.
NVIDIA Nemotron 3 Nano 30B A3B BF16
NVIDIA · 31.6B · runs from 19.3 GB at Q4
NVIDIA Nemotron 3 Nano 30B A3B is a mixture-of-experts model with 31.6 billion total parameters but only around 3 billion active per token, giving it the intelligence of a much larger model with the speed of a small one. This BF16 version preserves full precision for maximum output quality. The MoE architecture makes this model especially interesting for local deployment. You get reasoning and instruction-following capabilities that punch well above what a traditional 3B model can deliver, while inference stays fast because only a fraction of the network fires for each token.
Hy MT2 7B
Tencent · 8.0B · runs from 7.3 GB at Q4
Hy-MT2-7B is Tencent's mid-size entry in the Hy-MT2 family of dedicated, "fast-thinking" multilingual translation models, alongside 1.8B and 30B-A3B (mixture-of-experts) siblings. Rather than being a general-purpose chat assistant, it is trained specifically for translation across 33 languages and to follow multilingual translation instructions, and the card reports it outperforming open models such as DeepSeek-V4-Pro and Kimi K2.6 in fast-thinking mode. Tencent also ships FP8 and GGUF quantized builds for on-device use with llama.cpp. At roughly 8 billion dense parameters, it fits on a single consumer GPU once quantized. Context length is 262,144 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2026, alongside the open-sourced IFMTBench instruction-following translation benchmark.
GPT OSS 20B
OpenAI · 20.9B · runs from 13.1 GB at Q4
GPT-OSS 20B is one of OpenAI's first open-source model releases, marking a historic shift in the company's approach to open weights. At 21.5 billion parameters it delivers strong general-purpose chat and reasoning capabilities informed by the research behind the GPT family, making it a compelling option for users who want OpenAI-grade quality in a locally deployable package. The model runs comfortably on a single high-end consumer GPU such as an RTX 4090 at 4-bit quantization, or on workstation cards with 24 GB or more of VRAM at higher precision. It occupies a practical middle ground between lightweight 7B models and resource-heavy 70B+ offerings.
Qwen2.5 VL 3B Instruct
Alibaba · 3.8B · runs from 3.2 GB at Q4
Qwen2.5 VL 3B Instruct is Alibaba's 3.8-billion-parameter vision-language model in the Qwen 2.5 lineup, built to process images and text together in a single conversation. It can describe images, answer questions about visual content, read charts and documents, and locate objects within a scene, making it a compact option for on-device or edge multimodal applications. Its small size means it runs comfortably on modest consumer GPUs, and even on laptops or lower-end hardware once quantized, without requiring a workstation-class card. The model supports a 128K token context window, enough for lengthy documents or extended visual conversations. It was published in January 2025 alongside the larger Qwen2.5-VL models, sharing the same architecture and vision encoder scaled down for lighter-weight, latency-sensitive deployments.
Llama 3.1 8B Instruct
Meta · 8.0B · runs from 7.9 GB at Q4
Meta Llama 3.1 8B Instruct is an 8-billion parameter instruction-tuned language model from Meta. Part of the Llama 3.1 release, it supports a 128K token context window and is fine-tuned for conversational use, tool calling, and general assistant tasks. Its compact size makes it well-suited for local deployment on modern consumer GPUs with 8GB or more of VRAM. Llama 3.1 8B Instruct delivers strong performance for its parameter class across benchmarks in reasoning, coding, and multilingual understanding. It is released under the Llama 3.1 Community License and is widely supported by inference frameworks such as llama.cpp, vLLM, and Ollama.
Qwen3 30B A3B Instruct 2507
Alibaba · 30.5B · runs from 19.4 GB at Q4
Qwen3 30B A3B Instruct 2507 is a July 2025 updated mixture-of-experts model from Alibaba with 30 billion total parameters but only around 3 billion active during inference. This MoE architecture gives it a remarkably small memory and compute footprint relative to its total parameter count, letting users run a model with broad knowledge on mid-range hardware. The 2507 instruct refresh improves alignment and instruction-following quality over the original release. Because only a fraction of the weights are active at any given time, this model can often run on a single consumer GPU with 8 GB or more of VRAM when quantized, making it an excellent choice for users who want strong chat performance without heavyweight hardware.
Llama 3.2 3B Instruct
Meta · 3.2B · runs from 4.1 GB at Q4
Meta Llama 3.2 3B Instruct is a 3-billion parameter instruction-tuned model from Meta's Llama 3.2 release, designed for efficient local inference on resource-constrained hardware. It supports a 128K token context window and is optimized for conversational AI, summarization, and general assistant tasks. Despite its small footprint, Llama 3.2 3B Instruct delivers competitive performance for its size class and can run on GPUs with as little as 4GB of VRAM when quantized. It is released under the Llama 3.2 Community License and is a practical choice for edge deployment and lightweight local inference.
MiMo V2.6 Distill Qwen 9B
Xiaomi · 9.4B · runs from 6.5 GB at Q4
MiMo V2.6 Distill Qwen 9B is Xiaomi's 9.4-billion-parameter agentic model, built by fine-tuning Alibaba's Qwen3.5-9B on data generated by Xiaomi's larger MiMo-V2.6-Pro and MiMo-V2.6-Flash models. It handles vision, code, and function-calling tasks, and is positioned as the practical local-hardware option next to Xiaomi's much larger frontier MiMo-V2.6 models, which are designed for hosted use. At under 10 billion parameters, it runs well on a single mainstream consumer GPU once quantized. The model offers a 262K token context window, suited to long agent traces or multi-file coding sessions. Published in September 2026, it is one of the newest releases in Xiaomi's MiMo line, distilling the larger models' agentic and coding behaviour into a size that fits ordinary hardware.
Qwen3.5 27B
Alibaba · 27.8B · runs from 17.9 GB at Q4
Qwen3.5 27B is a 27.8-billion-parameter dense model from Alibaba's Qwen team, built for both text and image input using the same hybrid linear/full-attention architecture as the rest of the Qwen3.5 family. As a vision-capable model it can describe and reason about images alongside prompts, suiting multimodal chat and document tasks. At this size, local inference calls for quantization and a single high-end 24GB-plus consumer or workstation GPU rather than budget hardware. It supports a 262K token context window for long documents and multi-turn conversations, and is released under the Apache 2.0 license for unrestricted commercial and research use. Published in late February 2026, it sits alongside a separate small-model tier (0.8B–9B) in the same family, giving developers a mid-sized, single-GPU-friendly dense option.
Gemma 3 27B IT
Google · 27.4B · runs from 21.7 GB at Q4
Google Gemma 3 27B IT is a 27.4-billion parameter multimodal instruction-tuned model from Google's Gemma 3 family. It supports both text and image inputs, making it one of the most capable openly available vision-language models for local inference. The model handles conversational AI, visual question answering, image description, and complex reasoning tasks across modalities. Gemma 3 27B IT requires a GPU with at least 24GB of VRAM for quantized inference, placing it within reach of high-end consumer cards like the RTX 4090. It uses a dense Transformer architecture with a large context window and benefits from Google's extensive pretraining pipeline. Released under the Gemma license.
GLM 4.7 Flash
Z.ai · 31.2B · runs from 19.9 GB at Q4
GLM-4.7-Flash is a Mixture-of-Experts language model from Z.ai, part of the GLM 4 family, with about 31 billion total parameters and roughly 3 billion active per token. It is a chat and agentic model, positioned by Z.ai as a lightweight option for coding, tool use, and multi-turn agent tasks, with configurable reasoning-effort settings and support for extended thinking on harder problems. It supports a 202,752-token context window and is released under the MIT license. At this size, 4-bit quantization needs roughly 18GB of memory, putting it within reach of a single high-end consumer GPU in the 16-24GB range.
Qwen3 0.6B
Alibaba · 752M · runs from 1.7 GB at Q4
Qwen3 0.6B is the smallest instruction-tuned model in Alibaba Cloud's Qwen 3 family, with approximately 752 million parameters. It is designed for ultra-lightweight deployment where minimal hardware resources are available, running comfortably on virtually any modern GPU or CPU-only setups. The model supports hybrid thinking mode despite its tiny footprint. While limited in reasoning depth compared to larger variants, Qwen3 0.6B handles basic chat, simple summarization, and lightweight instruction following. It is primarily useful for edge deployment, rapid prototyping, and experimentation where model size is a critical constraint. Released under the Apache 2.0 license.
Gemma 2 9B IT
Google · 9.2B · runs from 7.3 GB at Q4
Google Gemma 2 9B IT is a 9.2-billion parameter instruction-tuned model from Google's Gemma 2 series. It is a text-only chat model optimized for conversational tasks, instruction following, and general-purpose assistance. At release, it was recognized for delivering unusually strong performance relative to its parameter count. The model runs efficiently on consumer GPUs with 8-12GB of VRAM in quantized formats, making it accessible on mainstream hardware. It is a popular choice for local inference among users who want strong quality without the VRAM demands of larger models. Released under the Gemma license.
Qwen2 VL 7B Instruct
Alibaba · 8.3B · runs from 6.2 GB at Q4
Qwen2 VL 7B Instruct is Alibaba's 8.3-billion-parameter vision-language model from the original Qwen 2 generation, built to process images and video alongside text in a single conversation. It can describe images, answer visual questions, and reason over multi-image or video input, suiting it to multimodal chat and visual document tasks. At this parameter count, local inference is practical on a single mainstream or high-end consumer GPU once the weights are quantized, rather than requiring multi-GPU hardware. The model supports a 32K token context window, enough for moderate-length documents or multi-turn visual conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in August 2024, it was later superseded by Qwen2.5-VL-7B-Instruct, which its own model card lists as its successor.
LFM2.5 24B A2B
Liquid AI · 24.0B · runs from 15.0 GB at Q4
LFM2.5-24B-A2B is Liquid AI's largest and final model in the LFM2.5 family, a text-only reasoning model with 24 billion total parameters and about 2.45 billion active per token. It is aimed at fast, private agents that run on-device, and its hybrid design mixes short-convolution blocks with a smaller number of attention layers. Because only a couple of billion parameters fire per token, decoding is quick; the card cites 184 tokens per second on an Apple M5 Max. A Q4 GGUF takes roughly 14 GB, so it fits a 16 GB GPU or a 24 GB Mac comfortably. The context length is 131,072 tokens. The model always thinks before answering, and the card highlights agentic and tool-use performance, comparing it favorably with the larger Qwen3.6-35B-A3B. It supports 16 languages, including English, Chinese, Japanese, Spanish, and Arabic. It ships under the LFM 1.0 license, a custom license rather than Apache or MIT, so check its terms before commercial use.
Qwen3 1.7B
Alibaba · 2.0B · runs from 3.4 GB at Q4
Qwen3 1.7B is a 1.7-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 3 series. It is a lightweight model designed for deployment on minimal hardware, including low-VRAM GPUs and even CPU-only configurations with acceptable latency. Despite its compact size, it supports hybrid thinking mode and handles basic conversational tasks, simple question answering, and text generation. The model is useful for edge deployment, embedded applications, and scenarios where fast inference with minimal resource consumption is the priority. It represents a significant quality improvement over Qwen 2.5 at the sub-2B scale. Released under the Apache 2.0 license.
Gemma 3 1B IT
Google · 1000M · runs from 2.1 GB at Q4
Google Gemma 3 1B IT is a 1-billion parameter instruction-tuned model from Google's Gemma 3 family. It is an ultra-compact text-only chat model designed for deployment on minimal hardware, including low-VRAM GPUs and edge devices. The model handles basic conversational tasks, simple instruction following, and lightweight text generation. It can run on virtually any modern GPU and even on CPU-only setups with acceptable latency. Released under the Gemma license.
Qwen3 32B
Alibaba · 32.8B · runs from 22.6 GB at Q4
Qwen3 32B is the flagship dense model in Alibaba Cloud's Qwen 3 series, with 32 billion parameters. It is instruction-tuned for chat and delivers strong performance across reasoning, coding, mathematics, and multilingual tasks. Qwen3 32B supports a hybrid thinking mode that allows the model to engage in extended chain-of-thought reasoning or respond quickly depending on the task, giving users flexibility between depth and speed. The model requires a GPU with at least 24GB of VRAM for quantized inference, placing it within reach of high-end consumer cards like the RTX 4090. It represents a significant generational improvement over Qwen 2.5 in both instruction following and knowledge breadth. Released under the Apache 2.0 license.
Ling 3.0 Tiny
Inclusion AI · 7.9B · runs from 5.5 GB at Q4
Ling 3.0 Tiny is InclusionAI's smallest mixture-of-experts model in Ant Group's Ling 3.0 family, totaling roughly 7.9 billion parameters with about 1.4 billion active per token. It shares the family's hybrid-linear-attention MoE architecture, also used at far larger scale in Ling-3.0-Flash, scaled down here for lightweight chat and agent use. Only the active parameters compute per token, so it responds quickly while still needing its full parameter set in memory; under 8 billion total, it runs on a mainstream consumer GPU once quantized. The model provides a 131K token context window for extended conversations or agent traces. It is released under the MIT license, allowing unrestricted commercial and research use. Published in August 2026, it extends Ant Group's Ling 3.0 lineup, built for efficient agent workloads.
Qwen2.5 Coder 14B Instruct
Alibaba · 14.8B · runs from 12.4 GB at Q4
Qwen2.5 Coder 14B Instruct is a 14.8B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 1.5B Instruct
Alibaba · 1.5B · runs from 1.7 GB at Q4
Qwen2.5 1.5B Instruct is a 1.5-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 2.5 series. It is a lightweight model suitable for deployment on minimal hardware, including low-VRAM GPUs and even CPU-only setups with acceptable latency. It supports a 128K token context window. The model handles basic conversational tasks, simple question answering, and text generation. While limited in reasoning depth compared to larger variants, it is useful for applications where fast response times and minimal resource consumption are priorities. Released under the Apache 2.0 license.
Spark X2.5 4B
XHToken · 4.1B · runs from 4.3 GB at Q4
Spark X2.5 4B is a 4.1B-parameter open language model from XHToken. It supports a context window of up to 1,048,576 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
DeepSeek R1 0528 Qwen3 8B
DeepSeek · 8.2B · runs from 7.6 GB at Q4
With about 8 billion parameters, DeepSeek R1 0528 Qwen3 8B is a dated May 2025 release in DeepSeek's R1 line, built on a Qwen3-based architecture rather than DeepSeek's own base model. It supports explicit step-by-step reasoning, generating intermediate reasoning traces before producing a final answer, which sets it apart from standard instruction-tuned chat models that answer directly. Its modest size means it runs comfortably on modest consumer GPUs, making it accessible for local experimentation with reasoning-style outputs without specialized hardware. It offers a 128K token context window, ample for most reasoning and coding tasks, and is released under the MIT license, allowing broad local and commercial use.
Qwen2.5 0.5B Instruct
Alibaba · 494M · runs from 0.8 GB at Q4
Qwen2.5 0.5B Instruct is the smallest instruction-tuned model in Alibaba Cloud's Qwen 2.5 family, with just 494 million parameters. It is designed for ultra-lightweight deployment scenarios where minimal hardware resources are available, running comfortably on virtually any modern GPU or even CPU-only configurations. Despite its tiny footprint, the model supports a 128K token context window and can handle basic chat, simple summarization, and lightweight instruction following. It is primarily useful for edge deployment, experimentation, and prototyping where model size is a critical constraint. Released under the Apache 2.0 license.
Qwen3.5 2B
Alibaba · 2.3B · runs from 1.9 GB at Q4
Qwen3.5 2B is a 2.3-billion-parameter model from Alibaba's Qwen team, one of the smaller entries in the Qwen3.5 lineup (0.8B–9B) built to handle text and image input together. It uses a hybrid architecture mixing linear-attention layers with periodic full-attention layers to keep long-context inference efficient. As a vision-capable model it can read and reason about images alongside written prompts, and its small size lets it run on a laptop or even a phone once quantized, without a dedicated GPU. It supports an unusually long 262K token context window for its size. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and was published in late February 2026 as part of Alibaba's push toward compact, natively multimodal edge models.
MiniCPM V 4.7 35B A3B
OpenBMB · 35.2B · runs from 22.1 GB at Q4
MiniCPM-V 4.7 35B-A3B is OpenBMB's vision-language model and the first mixture-of-experts entry in the MiniCPM-V series, with about 35 billion total parameters and roughly 4 billion active per token. It reads single images, multiple images, and video, and is built with phones and other edge devices in mind: the card describes a scheduling technique and layer-wise quantization that let it run on an iPhone 17 Pro Max at over 13 tokens per second. On a PC, a Q4 build fits a 24 GB GPU or a 32 GB unified-memory machine, and the small active set keeps decoding fast. The context window is 262,144 tokens. It is released under the Apache 2.0 license, so commercial use is unrestricted. The vision encoder compresses visual tokens early and supports mixed 4x and 16x compression, letting you trade image detail for speed. It works with llama.cpp, Ollama, vLLM, and SGLang, and GGUF, AWQ, GPTQ, and FP8 variants are available.
Spark X2.6 4B
XHToken · 4.1B · runs from 4.3 GB at Q4
Spark-X2.6-4B is a compact 4.1-billion-parameter dense language model from XHToken, aimed at on-device agents and everyday real-world tasks. It is the post-trained version of Spark-X2.6-4B-Base and keeps exactly the same architecture as Spark-X2.5-4B, with XHToken crediting improved post-training for gains in coding, agentic tool use, long-context handling, multilingual work and reasoning. It supports English and Chinese and has three configurable thinking levels (none, low and high), so an application can trade reasoning quality against latency. XHToken says it matches or exceeds some 9B-class models on several benchmarks. Context length is 1,048,576 tokens, though XHToken notes that using the full window needs plenty of device memory and recommends lowering the limit when necessary. It is released under the Apache 2.0 license, permitting commercial use, and was published in October 2026. At 4.1 billion parameters it is small enough for a laptop, a mid-range consumer GPU or a recent phone-class device once quantized, and keeping the context window modest keeps memory use low.
Unlimited OCR
Baidu · 3.3B · runs from 3.3 GB at Q4
Unlimited-OCR is Baidu's 3.3-billion-parameter vision-language model for OCR and document parsing, built to extend DeepSeek-OCR's approach further. It pairs a vision encoder with a Mixture-of-Experts decoder on DeepSeek's architecture, routing to 6 of 64 experts per token plus 2 shared experts, so only about 1.1 billion parameters activate per token even though every expert must still fit in memory. Its distinguishing idea, "one-shot long-horizon parsing," extends optical context compression to longer documents processed in a single pass. It is small enough to run on a single consumer GPU once quantized. Context length is 32,768 tokens, oriented around page and document parsing. It is released under the MIT license, a highly permissive option for commercial use. Published in June 2026, it has quickly picked up community support for vLLM inference and ms-swift fine-tuning.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.