All LLM Models
Browse 1475 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Seed OSS 36B Instruct
ByteDance-Seed · 36.2B · runs from 10.5 GB
53.7K
505Seed OSS 36B Instruct is a 36.2B-parameter open language model from ByteDance-Seed in the Seed-OSS family. It supports a context window of up to 524,288 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nous Hermes 2 Mixtral 8x7B DPO
Nous Research · 46.7B · runs from 19.8 GB
Nous-Hermes-2-Mixtral-8x7B-DPO is Nous Research's instruction-tuned chat model built on top of Mistral AI's Mixtral-8x7B-v0.1 Mixture-of-Experts base, trained through supervised fine-tuning followed by direct preference optimization (DPO) on over a million largely GPT-4-generated examples plus other curated open data. It uses the ChatML prompt format for structured multi-turn dialogue and system prompts, and improved on GPT4All, AGIEval, and BigBench benchmarks over both the base Mixtral model and Mistral's own Mixtral-Instruct. With 8 experts and 2 active per token, roughly 12.9 billion active out of 46.7 billion total, it needs a high-end consumer GPU or multi-GPU setup once quantized. Context length is 32,768 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in January 2024, alongside an SFT-only sibling release for comparison.
Hunyuan 7B Instruct
Tencent · 7.5B · runs from 2.6 GB
Hunyuan 7B Instruct is a 7.5B-parameter open language model from Tencent in the Hunyuan family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Moondream2
vikhyatk · 1.9B · runs from 0.9 GB
Moondream2 is a compact 1.9-billion-parameter vision-language model from the Moondream project, published under developer Vikhyat Korrapati's vikhyatk account, designed for lightweight image understanding rather than open-ended chat. It handles image captioning, visual question answering, object detection, and point-based visual grounding, useful for document pipelines and robotics perception where a full-size multimodal model is unnecessary. Its small size means it runs comfortably on modest consumer GPUs and even CPU-only or edge setups. Because Moondream2 focuses on single-image and short-prompt tasks, it does not advertise an extended context window like larger chat models. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. First published in March 2024, it remains a popular choice for on-device and edge vision applications, with built-in object detection and pointing alongside captioning.
Mistral Large 3 675B Instruct 2512
Mistral AI · 675B · runs from 204.2 GB
Mistral-Large-3-675B-Instruct-2512 is Mistral AI's flagship instruction-tuned model, a multimodal granular Mixture-of-Experts system pairing a roughly 673-billion-parameter, about 39-billion-active-parameter language backbone with a 2.5-billion-parameter vision encoder, for around 675 billion total and 41 billion active parameters overall. It handles vision alongside text, supports dozens of languages, and is built for agentic use with native function calling and JSON output, aimed at long-document understanding, coding, and enterprise knowledge work rather than dedicated step-by-step reasoning. This is a frontier-scale model that requires a full multi-GPU server node to run, even quantized. Context length is 262,144 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in November 2025, as the third generation of Mistral's Large model line, alongside FP8, NVFP4, and BF16 weight releases.
Falcon H1 7B Instruct
TII UAE · 7.6B · runs from 2.6 GB
Falcon H1 7B Instruct is TII UAE's instruction-tuned chat model built on Falcon-H1-7B-Base, using a hybrid architecture that combines standard Transformer attention with Mamba state-space layers in a causal decoder-only design. That hybrid-head approach is meant to keep long-context inference efficient while retaining Transformer-level quality, and the model supports 18 languages including Arabic, Chinese, Japanese, and several European languages. On released benchmarks it beats similarly sized Qwen2.5-7B, Llama-3.1-8B, and Falcon3-7B/10B on most general knowledge, science, and instruction-following tasks. At 7.6 billion parameters it fits a single consumer GPU. Context length is 262,144 tokens. It is released under the Falcon LLM License, a custom TII license that permits commercial use subject to an acceptable-use policy and an attribution requirement in derivative works, and was published in May 2025.
Pythia 410M Deduped
EleutherAI · 506M · runs from 0.1 GB
Pythia 410M Deduped is a 506M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GLM 4.7
Z.ai · 358.3B · runs from 99.2 GB
GLM 4.7 is an earlier generation of Zhipu AI's GLM foundation model series, featuring a mixture-of-experts architecture with approximately 358 billion total parameters. It delivers strong performance on reasoning, language understanding, and bilingual Chinese-English tasks while being significantly more manageable to run locally than its GLM 5 successor. For users with multi-GPU setups, GLM 4.7 offers a practical balance between capability and hardware requirements within the GLM model family.
Intern S2 397B
InternLM · 403.4B · runs from 111.4 GB
Intern-S2-397B is Shanghai AI Laboratory's multimodal foundation model, built for scientific intelligence and long-horizon agent tasks. It is a Mixture-of-Experts model with roughly 397 billion parameters, using a vision-language pretraining approach that learns directly from raw pages of scientific literature, jointly modeling text and visual relationships without an intermediate parsing step. It supports a thinking mode for deeper reasoning, enabled by default, and was trained with reinforcement learning across more than 20 scientific domains. It was evaluated with up to a 256K token context for text reasoning and 64K for multimodal inputs, and is released under the Apache 2.0 license. At roughly 397 billion parameters, running it locally needs multi-GPU server-class hardware; most users will use a hosted endpoint instead.
Deepseek Coder 33B Base
DeepSeek · 33.3B · runs from 14.6 GB
DeepSeek-Coder-33B-Base is DeepSeek's 33.3-billion-parameter base (pretrained, not instruction-tuned) code model, trained from scratch on 2 trillion tokens (87% code, 13% natural language in English and Chinese) using grouped-query attention, project-level code context, and a fill-in-the-blank training objective for project-level completion and infilling. It is intended as a foundation for further fine-tuning or direct code completion and infilling use, not conversational use; the instruction-tuned DeepSeek-Coder-33B-Instruct is built on top of it. At release, the Deepseek Coder family achieved state-of-the-art open-model results on HumanEval, MultiPL-E, MBPP, DS-1000, and APPS. At 33.3 billion parameters, it needs a high-end consumer GPU or a multi-GPU setup once quantized. Context length is 16,384 tokens. Model weights are released under DeepSeek's custom model license, which permits commercial use. It was published in October 2023, alongside 1.3B, 5.7B, and 6.7B siblings and the corresponding instruct checkpoint.
Llama 4 Maverick 17B 128E Instruct
Meta · 401.6B · runs from 121.5 GB
Llama 4 Maverick is Meta's instruction-tuned, natively multimodal mixture-of-experts model, with roughly 401.6 billion total parameters and about 17 billion active per token routed across 128 experts. It accepts multilingual text and image input and produces multilingual text and code output, using early fusion to integrate vision and language from pretraining rather than bolting on a separate vision encoder; Meta trained it on roughly 22 trillion tokens of multimodal data with an August 2024 knowledge cutoff. It is built for assistant-style chat and visual reasoning tasks. Despite its relatively small active-parameter count, its total size means it needs a multi-GPU server-class setup to run even once quantized. Context length is up to 1,000,000 tokens. It is released under the Llama 4 Community License, a custom license that is free for most commercial and research use but requires companies with more than 700 million monthly active users to request separate permission from Meta. It was published in April 2025.
Agnes 3.0 Flash
Agnes-AI · 33.1B · runs from 14.9 GB
Agnes-3.0-Flash Preview is Agnes AI's 33-billion-parameter open-weight multimodal model offering competitive reasoning, coding, and instruction-following at lower hardware cost than flagship-class models; it understands text, images, and video, supports tool calling, and lets developers dial reasoning effort up or down. This open-weight Preview checkpoint is distinct from the larger production/API "Agnes 3.0 Flash" model on third-party leaderboards, which uses a different checkpoint with a 1-million-token context window, so its benchmark results don't transfer to these weights. Architecturally it is a hybrid-attention decoder where three of every four layers use a recurrent gated-delta-rule mechanism instead of standard attention, so only 18 of its 72 layers hold a KV cache that grows with context. At 33 billion dense parameters, it needs a high-end consumer GPU or multi-GPU setup once quantized. Context length is 262,144 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2026.
Magistral Small 2506
Mistral AI · 23.6B · runs from 7.2 GB
Magistral Small 2506 is Mistral AI's small, efficient reasoning model, built on the 24-billion-parameter Mistral Small 3.1 and further trained with supervised fine-tuning on Magistral Medium's reasoning traces plus reinforcement learning. It produces long chains of reasoning before answering and supports dozens of languages, making it suited to multi-step problem solving rather than simple chat. The architecture supports up to 128K tokens, but Mistral recommends keeping usage around 40K since output quality can degrade past that point. Released under the Apache 2.0 license, it is compact enough to fit on a single RTX 4090 or a 32 GB RAM Mac once quantized to 4-bit, in line with what a 24-billion-parameter model needs at that precision.
Qwen2.5 Coder 3B
Alibaba · 3.1B · runs from 1.4 GB
Qwen2.5-Coder-3B is Alibaba's 3.1-billion-parameter base (pretrained, not instruction-tuned) code language model, one of six sizes in the Qwen2.5-Coder family built on Qwen2.5-3B. It was pretrained on 5.5 trillion tokens of source code, text-code grounding data, and synthetic data, and is intended as a foundation for further fine-tuning, reinforcement learning, or fill-in-the-middle code-completion tasks rather than direct chat use; Alibaba explicitly advises against using it for conversations as-is. The family's larger 32B model is described as matching GPT-4o-level coding ability, though this 3B checkpoint targets lightweight, resource-constrained deployment. At just over 3 billion parameters, it runs easily on a single modest consumer GPU or even a CPU. Context length is 32,768 tokens. It is released under the Qwen Research License, a non-commercial license restricted to research and evaluation use; commercial use requires a separate license from Alibaba Cloud. It was published in November 2024.
Pythia 6.9B
EleutherAI · 7.0B · runs from 2.1 GB
Pythia 6.9B is a 7.0B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 4 Reasoning Plus
Microsoft · 14.7B · runs from 4.8 GB
Phi 4 Reasoning Plus is a 14.7B-parameter open language model from Microsoft in the Phi 4 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Pythia 160M Deduped
EleutherAI · 213M · runs from 0.1 GB
Pythia 160M Deduped is a 213M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Olmo 3 7B Instruct
Allen AI · 7.3B · runs from 3.4 GB
OLMo 3 7B Instruct is an instruction-tuned language model from the Allen Institute for AI, built as part of their Open Language Model initiative. Like all OLMo releases, it comes with fully open training data, code, and intermediate checkpoints, setting a high standard for reproducibility and scientific transparency in the LLM space. At roughly 7 billion parameters, this model delivers competitive performance on instruction following, reasoning, and general knowledge tasks while remaining runnable on consumer GPUs with 8 GB or more of VRAM. It is an excellent choice for users who value open science and want a capable, well-documented model for local chat and assistant applications.
GLM 4.5V
Z.ai · 107.7B · runs from 30.1 GB
GLM-4.5V is Z.ai's vision-language model built on its GLM-4.5-Air text foundation, a mixture-of-experts model with roughly 14.3 billion active parameters out of about 107.7 billion total. It continues the technical approach of GLM-4.1V-Thinking and covers image, video, and long-document understanding, GUI agent operation, complex chart parsing, and visual grounding, reaching state-of-the-art results among similarly sized models on 42 public vision-language benchmarks. A Thinking Mode toggle lets it trade fast responses for deeper step-by-step reasoning, mirroring the same switch in the GLM-4.5 text model. Given its scale, it needs a multi-GPU workstation once quantized. Context length is 65,536 tokens. It is released under the MIT license, permitting unrestricted commercial and research use. It was published in August 2025, succeeding the smaller GLM-4.1V-9B-Thinking model.
MiniCPM O 2 6
OpenBMB · 8.7B · runs from 3.4 GB
MiniCPM-o 2.6 is OpenBMB's 8.7-billion-parameter omni-modal model, combining a SigLip-400M vision encoder, a Whisper-medium audio encoder, a ChatTTS speech decoder, and a Qwen2.5-7B language backbone trained end to end. Unlike a pure vision-language model, it takes images, video, and audio as input and can generate spoken output too, supporting real-time bilingual speech conversation, voice cloning, and live-streaming understanding. At this size it is practical on a single high-end consumer GPU once quantized. Context length is 32,768 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in January 2025. Its distinguishing feature is the end-to-end omni-modal architecture, using time-division multiplexing that lets it process continuous video and audio streams and reply with synthesized speech, not just text.
GPT Neox 20B
EleutherAI · 20.7B · runs from 6.3 GB
GPT Neox 20B is a 20.7B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
UI TARS 1.5 7B
ByteDance-Seed · 8.3B · runs from 2.7 GB
UI-TARS-1.5-7B is ByteDance's 8.3-billion-parameter vision-language model, built on a Qwen2.5-VL-7B foundation and tuned as a GUI and computer-use agent rather than a chatbot. It reasons through its thoughts before acting, using reinforcement learning to control desktop and browser interfaces, play games, and complete multi-step tasks from screenshots. ByteDance evaluates it against agents like OpenAI's CUA and Claude on benchmarks such as OSWorld and Windows Agent Arena. At this size, local inference is practical on a single mainstream-to-high-end consumer GPU once quantized. It supports a 128,000 token context window, useful for long action histories in agent tasks. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in April 2025, it builds on the original UI-TARS architecture with added inference-time reasoning scaling.
Gemma 4 E4B
Google · 8.0B · runs from 3.9 GB
Gemma 4 E4B is Google DeepMind's second-smallest model in the Gemma 4 family, a dense architecture with roughly 8 billion total parameters, of which Google describes about 4.5 billion as its effective footprint at inference. This is the pretrained base checkpoint rather than an instruction-tuned model, meant for fine-tuning rather than direct chat use. Like the rest of the family it is multimodal — text, image, and natively audio at this size — targeting efficient on-device deployment on laptops and higher-end phones. It runs comfortably on a single consumer GPU, or on-device once quantized. It supports a 131,072 token context window. It carries Google's Gemma 4 license terms, published as Apache 2.0 on Hugging Face with additional Gemma-specific usage terms linked from the card. Published in March 2026, it sits between the E2B on-device model and the larger 12B, 26B-A4B, and 31B tiers.
Qwen3.6 35B A3B DFlash
z-lab · 35B · runs from 15.2 GB
Qwen3.6 35B A3B DFlash is a 35B-parameter open language model from z-lab in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Hermes 4 70B
Nous Research · 70.6B · runs from 31.0 GB
Hermes 4 70B is a 70.6B-parameter open language model from Nous Research in the Hermes family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Yi 1.5 9B Chat
01.AI · 8.8B · runs from 4.1 GB
Yi-1.5-9B-Chat is 01.AI's second-generation 9-billion-parameter bilingual (English/Chinese) chat model, continually pretrained from the original Yi series on a further 500 billion high-quality tokens and then fine-tuned on 3 million diverse instruction samples. Compared with the original Yi, Yi-1.5 delivers stronger coding, math, and reasoning while keeping the same language understanding and commonsense reasoning, and 01.AI reports it as the top performer among similarly sized open models on its benchmark suite. At 9B parameters it fits comfortably on a single consumer GPU, or a much smaller card once quantized. Context length is 4,096 tokens; separate 16K- and 32K-context variants of the same model are also available for longer documents. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2024.
Llama 3 3 Nemotron Super 49B V1
NVIDIA · 49.9B · runs from 15.1 GB
Llama 3 3 Nemotron Super 49B V1 is a 49.9B-parameter open language model from NVIDIA in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Meta Llama 3.1 8B Instruct Abliterated
mlabonne · 8.0B · runs from 3.3 GB
Meta Llama 3.1 8B Instruct Abliterated is a 8.0B-parameter open language model from mlabonne in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
MiniMax M2
MiniMax · 228.7B · runs from 63.5 GB
MiniMax-M2 is a compact Mixture-of-Experts model from MiniMax built specifically for coding and agentic workflows, with 230 billion total parameters but only about 10 billion activated per token. It is designed for multi-file edits, coding-run-fix loops, and long-horizon tool use across shells, browsers, and code runners, aiming to keep the latency and cost of agentic use manageable despite its overall size. It supports a 196,608-token context window and is released under a custom, MIT-derived license. Because all experts must stay resident in memory, 4-bit quantization needs roughly 132GB, so local deployment calls for a multi-GPU server.
EXAONE 3.5 7.8B Instruct
LGAI-EXAONE · 7.8B · runs from 2.7 GB
EXAONE 3.5 7.8B Instruct is a 7.8B-parameter open language model from LGAI-EXAONE in the EXAONE family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.