All LLM Models
Browse 1699 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Model List
Ling 3.0 Flash
Inclusion AI · 127.5B · runs from 77.6 GB at Q4
9.5K
437Ling 3.0 Flash is a 127.5-billion-parameter Mixture-of-Experts language model from InclusionAI, Ant Group's open-model initiative, with only about 5.6 billion parameters active per token, keeping per-token compute low even though all expert weights must still fit in memory. It is built for chat and agentic workloads such as tool use and multi-step tasks, blending Ant's fast "Flash" line with reasoning techniques from its "Ring" series. Local deployment needs multiple high-VRAM GPUs or a large unified-memory machine. It supports a 262K token context window for long documents and agent sessions. It is released under the MIT license, a permissive option for commercial and research use. Published in August 2026, it activates only a small fraction of its experts per token to approach larger-model quality at lower compute.
Llama 3.3 70B Instruct
Meta · 70.6B · runs from 51.3 GB at Q4
Meta Llama 3.3 70B Instruct is a 70-billion parameter large language model from Meta, released as part of the Llama 3.3 generation. It is an instruction-tuned model optimized for dialogue and chat use cases, offering strong performance across reasoning, coding, and multilingual tasks. Llama 3.3 70B delivers quality competitive with much larger models while remaining feasible to run on high-end consumer or workstation GPUs with sufficient VRAM. The model uses a grouped-query attention architecture with a 128K token context window and was trained on a massive multilingual corpus. It is released under the Llama 3.3 Community License, making it one of the most capable openly available models for local inference.
Jan v3 4B Base Instruct
janhq · 4.4B · runs from 4.5 GB at Q4
Jan v3 4B Base Instruct is a 4.4B-parameter open language model from janhq. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Kimi K2.5
Moonshot AI · 1026.9B · runs from 617.6 GB at Q4
Kimi K2.5 is an earlier release in Moonshot AI's Kimi K2 series, sharing the roughly 1-trillion-parameter scale of later checkpoints in the line. Released at the start of 2026, it accepts image input alongside text, giving it multimodal capability on top of standard chat use. Its context window reaches 256K tokens, enough for lengthy documents or long conversation histories, and it is released under a custom license outside the standard open-source terms. Because of its size, Kimi K2.5 needs multi-GPU or server-class hardware; most people will use it through a hosted endpoint rather than running it themselves.
LFM2.5 2.6B
Liquid AI · 2.7B · runs from 2.3 GB at Q4
LFM2.5-2.6B is Liquid AI's on-device model in the LFM2.5 family, a hybrid architecture mixing gated short-convolution blocks with grouped-query attention layers, at about 2.7 billion parameters. It is post-trained for agentic workloads, tool use, data extraction, retrieval-augmented generation, and long-context tasks, though the publisher does not recommend it for heavy agentic coding or knowledge-intensive question answering. It supports a 131K token context window and is released under Liquid AI's custom LFM1.0 license. At around 2.7 billion parameters, it runs easily on almost any modern laptop or consumer GPU at 4-bit quantization, and the publisher reports it using well under 2.5 GB of memory during on-device inference.
Gemma 4 31B IT Qat Q4 0 Unquantized
Google · 31.3B · runs from 21.7 GB at Q4
Gemma 4 31B IT Qat Q4 0 Unquantized is a 31.3B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
LFM3 1.7B
Liquid AI · 1.7B · runs from 2.1 GB at Q4
LFM3-1.7B is Liquid AI's text-only language model for local agents and specialized applications, and the smallest instruct model in the LFM3 family. It has 1.73 billion parameters and a tri-hybrid architecture of 24 layers: 13 gated short convolutions, 5 sliding-window attention layers, and 6 global attention layers. It is post-trained for agentic harnesses such as Hermes Agent, Pi, and OpenClaw, with tool calling and structured outputs, and is meant to be fine-tuned to a specific domain on a single GPU. The publisher advises using retrieval or search for knowledge-heavy tasks. Context length is 256K tokens, input plus output. It is released under the LFM Open License v1.0, a custom license whose terms are in the repository, and was published in October 2026. At 4-bit precision it needs about 1 GB of memory, so it runs on almost any modern phone, laptop, or consumer GPU.
Granite 4.2 8B
IBM · 8.8B · runs from 8.3 GB at Q4
Granite 4.2 8B is a 8.8B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Kimi K3
Moonshot AI · 2779.9B · runs from 1670.0 GB at Q4
Kimi K3 from Moonshot AI is an enormous model, weighing in at roughly 2.8 trillion parameters, among the largest open-weight releases in this catalogue. It accepts image input alongside text, supporting multimodal chat and visual reasoning in addition to standard dialogue. At this scale, local deployment is not realistic for most users; it needs multi-GPU or server-class hardware, and most people will access it through a hosted endpoint rather than running it on their own machines. Kimi K3 supports a 1M token context window, allowing it to process very long documents, codebases, or conversation histories in one pass. It was released on June 13, 2026, under its own custom license terms rather than a standard Apache or MIT license, so users should review the license text before commercial use.
Qwen2.5 Coder 1.5B Instruct
Alibaba · 1.5B · runs from 1.7 GB at Q4
Qwen2.5 Coder 1.5B Instruct is a 1.5B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
LFM2 24B A2B
Liquid AI · 23.8B · runs from 14.9 GB at Q4
LFM2 24B A2B is a 23.8B-parameter open language model from Liquid AI in the LFM2 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
LFM3 Omni 2B
Liquid AI · 2.2B · runs from 2.4 GB at Q4
LFM3-Omni-2B is Liquid AI's first omni-modal model in the LFM3 family: a single end-to-end model that understands text, images, video, and audio and answers in text, speech, or both interleaved. It has about 2.2 billion parameters, built from the LFM3 backbone with an LFM3 vision encoder, a causal audio encoder, and an audio tokenizer. One model can stand in for separate speech recognition, vision-language, text, and speech synthesis models, and it can watch and listen to a video with sound at the same time. Liquid AI lists a 32K-token context for omni sessions. It is released under the LFM Open License v1.0, a custom license whose terms are in the repository, and was published in October 2026. It is designed to run efficiently on phones, laptops, and other edge hardware, so a single consumer GPU is more than enough.
Gemma 3 12B IT
Google · 12.2B · runs from 10.9 GB at Q4
Google Gemma 3 12B IT is a 12-billion parameter multimodal instruction-tuned model from Google's Gemma 3 series. It supports both text and image inputs, offering vision-language capabilities at a more accessible size point than the 27B variant. Gemma 3 12B IT runs on consumer GPUs with 12-16GB of VRAM in quantized formats, making it a practical choice for local multimodal inference without requiring top-tier hardware. Released under the Gemma license.
KAT Coder V2.5 Dev
Kwaipilot · 34.7B · runs from 21.3 GB at Q4
KAT Coder V2.5 Dev is a 34.7B-parameter open language model from Kwaipilot. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Devstral Small 2 24B Instruct 2512
Mistral AI · 24.0B · runs from 18.1 GB at Q4
Devstral Small 2 24B Instruct is Mistral AI's dense 24-billion-parameter model for agentic software-engineering work, fine-tuned to follow instructions for chat, coding agents, and tool-heavy workflows. Built on the same architecture as Ministral 3, it adds vision capabilities for analyzing images alongside code and text, and its publisher designed it specifically to be lightweight enough for local, on-device use rather than requiring a large server. It supports a context window of roughly 384,000 tokens and is released under the Apache 2.0 license. Mistral notes it is light enough to run on a single RTX 4090 or a Mac with 32GB of RAM, consistent with its 4-bit memory needs of around 14GB.
MiniCPM V 4.7 1B
OpenBMB · 1.6B · runs from 1.6 GB at Q4
MiniCPM-V 4.7 1B is the smallest model in OpenBMB's MiniCPM-V 4.7 line, a vision-language model with roughly 1.5 billion parameters that handles single images, multiple images, and video. It is designed for phones, laptops, and other resource-limited devices, and the card says it substantially outperforms Qwen3.5 0.8B in both overall score and inference efficiency. At this size it runs on almost any hardware, including CPU-only machines, with a Q4 build needing around 1 GB of memory plus the vision encoder. The context window is 262,144 tokens. It is released under the Apache 2.0 license, which allows commercial use. Visual tokens are compressed early inside the vision encoder, and the model supports mixed 4x and 16x compression so you can favor detail or speed. It runs in llama.cpp, Ollama, vLLM, and SGLang, with GGUF, AWQ, GPTQ, and FP8 builds, and it has mobile support for iOS, Android, and HarmonyOS.
North Mini Code 1.0
Cohere · 30.5B · runs from 19.4 GB at Q4
North Mini Code 1.0 is Cohere's first open-weight coding model, a mixture-of-experts design with about 30.5 billion total parameters and roughly 3.3 billion active per token. It is tuned for agentic coding: tool calling, terminal workflows, and software-engineering tasks rather than general chat. Only the active parameters compute per token, so it decodes quickly for its size, though all weights must fit in memory; it runs on a single high-end consumer GPU once quantized. The model supports a 500,000 token context window, useful for large codebases or long agent transcripts. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in June 2026 as the debut release in Cohere's North family, it is designed to run on a single H100-class GPU in FP8.
Phi 4 Mini Instruct
Microsoft · 3.8B · runs from 4.8 GB at Q4
Microsoft Phi 4 Mini Instruct is a 3.8-billion parameter instruction-tuned model from Microsoft Research's Phi 4 family. It applies the Phi series' data-centric training philosophy to a compact model, delivering strong performance in coding, reasoning, and chat tasks relative to its small footprint. The model runs on consumer GPUs with as little as 4-6GB of VRAM when quantized, making it accessible on mainstream and even entry-level hardware. Released under the MIT license.
Kimi K2 Thinking
Moonshot AI · 1026.4B · runs from 617.3 GB at Q4
Kimi-K2-Thinking is Moonshot AI's reasoning-and-tool-use agent model, trained end-to-end to interleave chain-of-thought reasoning with function calls across long sequences of tool invocations. It is a Mixture-of-Experts model with about 1 trillion total parameters and roughly 32 billion active per token, and unusually it is natively trained with INT4 quantization-aware training, so the released weights ship already quantized. It has a 256K token context window and is released under a modified MIT license. Because it is natively INT4, its memory footprint is lower than a typical trillion-parameter model, but it is still large enough that local use needs server-class or multi-GPU hardware; most people will reach it through a hosted API instead.
DeepSeek R1 Distill Qwen 7B
DeepSeek · 7.6B · runs from 5.8 GB at Q4
DeepSeek R1 Distill Qwen 7B compresses the reasoning techniques from DeepSeek's full R1 model into a compact 7.6 billion parameter dense model built on the Qwen 2.5 architecture. Despite its small footprint, it demonstrates surprisingly capable step-by-step reasoning on math and logic problems that would stump many models several times its size. This is one of the most accessible reasoning models available for local use, fitting comfortably on GPUs with 6 GB or more of VRAM when quantized. It strikes a practical balance between genuine chain-of-thought reasoning ability and the hardware constraints of a typical consumer setup.
Qwen2.5 Coder 3B Instruct
Alibaba · 3.1B · runs from 2.8 GB at Q4
Qwen2.5 Coder 3B Instruct is a 3.1B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Spark X2.5 1.7B
XHToken · 1.7B · runs from 2.3 GB at Q4
Spark X2.5 1.7B is a 1.7B-parameter open language model from XHToken. It supports a context window of up to 1,048,576 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GLM 4.6V Flash
Z.ai · 10.3B · runs from 7.2 GB at Q4
GLM 4.6V Flash is a 10.3B-parameter open language model from Z.ai in the GLM 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 E2B IT Qat Q4 0 Unquantized
Google · 5.1B · runs from 3.5 GB at Q4
Gemma 4 E2B IT Qat Q4 0 Unquantized is a 5.1B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
S1 Mini
superwhisper · 752M · runs from 1.7 GB at Q4
S1 Mini is a 752M-parameter open language model from superwhisper. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 Omni 30B A3B Instruct
Alibaba · 35.3B · runs from 27.2 GB at Q4
Qwen3-Omni-30B-A3B-Instruct is Alibaba's flagship omni-modal model, a mixture-of-experts Thinker-Talker system that processes text, images, audio, and video and responds with real-time streaming text and speech. Its text backbone routes across 128 experts with 8 active per token, roughly 3 billion active parameters out of around 35 billion total, since it keeps both the "thinker" and the speech-generating "talker" plus its audio codec resident in memory. It supports 119 text languages, 19 spoken input languages, and 10 spoken output languages, and needs a high-end consumer GPU or multi-GPU setup once quantized. Its language backbone supports a 65,536 token context window. It is released under the Apache 2.0 license, and was published in September 2025 as the instruct counterpart to Qwen3-Omni-30B-A3B-Thinking, which drops the talker to focus on text output with chain-of-thought reasoning.
Inkling
Thinking Machines · 952.4B · runs from 575.0 GB at Q4
Inkling is the flagship model in Thinking Machines' Inkling family, a vision-language chat model with roughly 952 billion parameters. It accepts image input alongside text and supports general conversational use, and was released about two weeks ahead of the smaller Inkling Small model in the same line, in mid-July 2026. Inkling is distributed under the Apache 2.0 license, allowing free local and commercial use without restriction. Given its scale, it needs multi-GPU or server-class hardware to run at all; most users will access it through a hosted endpoint rather than running it on their own machines.
Huihui Qwen3.5 9B Abliterated
huihui-ai · 9.7B · runs from 6.6 GB at Q4
Huihui Qwen3.5 9B Abliterated is a 9.7B-parameter open language model from huihui-ai in the Qwen 3.5 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron Nano 9B v2
NVIDIA · 8.9B · runs from 6.0 GB at Q4
NVIDIA Nemotron Nano 9B v2 is a compact yet capable chat model from NVIDIA, packing 8.9 billion parameters into a size that runs comfortably on a wide range of consumer GPUs. Built on NVIDIA's Nemotron architecture, it delivers strong instruction-following and conversational performance while keeping VRAM requirements modest. This second-generation Nano model reflects NVIDIA's push to make high-quality language models accessible on local hardware. It's an excellent starting point for users who want a responsive, general-purpose assistant without needing top-tier GPU memory.
Mistral Small 3.2 24B Instruct 2506
Mistral AI · 24.0B · runs from 18.1 GB at Q4
Mistral-Small-3.2-24B-Instruct-2506 is a dense 24-billion-parameter vision-language model from Mistral AI, a minor refinement of Mistral-Small-3.1-24B-Instruct-2503. The update focuses on following precise instructions more reliably, cutting down on repetitive or runaway generations, and making function calling more robust, while still handling both text and image inputs for tasks like chart and document understanding. It supports a 131,072-token context window and is released under the Apache 2.0 license. At this size, 4-bit quantization needs roughly 14GB of memory, so it runs on a single consumer GPU in the 16-24GB range, such as an RTX 4090.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.