All LLM Models

Browse 1214 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Devstral Small 2505

Mistral AI · 23.6B · runs from 7.2 GB

Devstral Small 2505, internally called Devstral Small 1.0, is an agentic coding model built by Mistral AI in collaboration with All Hands AI, finetuned from Mistral-Small-3.1-24B-Base-2503 with its vision encoder removed, so it is text-only. It is designed to explore codebases and edit multiple files as a software engineering agent, and reached 46.8% on SWE-Bench Verified under the OpenHands scaffold, the top score among open models at release, ahead of GPT-4.1-mini and Claude 3.5 Haiku on the same benchmark. At 24 billion parameters it is explicitly built to be lightweight enough for a single consumer GPU or an Apple Silicon Mac, making it a real local-deployment option rather than a server-only model. Context length is 131,072 tokens (a 128k window). It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2025; a larger 123-billion-parameter Devstral 2 later succeeded it.

Seed OSS 36B Instruct

ByteDance-Seed · 36.2B · runs from 10.5 GB

Seed OSS 36B Instruct is a 36.2B-parameter open language model from ByteDance-Seed in the Seed-OSS family. It supports a context window of up to 524,288 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Nous Hermes 2 Mixtral 8x7B DPO

Nous Research · 46.7B · runs from 19.8 GB

Nous-Hermes-2-Mixtral-8x7B-DPO is Nous Research's instruction-tuned chat model built on top of Mistral AI's Mixtral-8x7B-v0.1 Mixture-of-Experts base, trained through supervised fine-tuning followed by direct preference optimization (DPO) on over a million largely GPT-4-generated examples plus other curated open data. It uses the ChatML prompt format for structured multi-turn dialogue and system prompts, and improved on GPT4All, AGIEval, and BigBench benchmarks over both the base Mixtral model and Mistral's own Mixtral-Instruct. With 8 experts and 2 active per token, roughly 12.9 billion active out of 46.7 billion total, it needs a high-end consumer GPU or multi-GPU setup once quantized. Context length is 32,768 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in January 2024, alongside an SFT-only sibling release for comparison.

Hunyuan 7B Instruct

Tencent · 7.5B · runs from 2.6 GB

Hunyuan 7B Instruct is a 7.5B-parameter open language model from Tencent in the Hunyuan family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Moondream2

vikhyatk · 1.9B · runs from 0.9 GB

Moondream2 is a compact 1.9-billion-parameter vision-language model from the Moondream project, published under developer Vikhyat Korrapati's vikhyatk account, designed for lightweight image understanding rather than open-ended chat. It handles image captioning, visual question answering, object detection, and point-based visual grounding, useful for document pipelines and robotics perception where a full-size multimodal model is unnecessary. Its small size means it runs comfortably on modest consumer GPUs and even CPU-only or edge setups. Because Moondream2 focuses on single-image and short-prompt tasks, it does not advertise an extended context window like larger chat models. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. First published in March 2024, it remains a popular choice for on-device and edge vision applications, with built-in object detection and pointing alongside captioning.

Falcon H1 7B Instruct

TII UAE · 7.6B · runs from 2.6 GB

Falcon H1 7B Instruct is TII UAE's instruction-tuned chat model built on Falcon-H1-7B-Base, using a hybrid architecture that combines standard Transformer attention with Mamba state-space layers in a causal decoder-only design. That hybrid-head approach is meant to keep long-context inference efficient while retaining Transformer-level quality, and the model supports 18 languages including Arabic, Chinese, Japanese, and several European languages. On released benchmarks it beats similarly sized Qwen2.5-7B, Llama-3.1-8B, and Falcon3-7B/10B on most general knowledge, science, and instruction-following tasks. At 7.6 billion parameters it fits a single consumer GPU. Context length is 262,144 tokens. It is released under the Falcon LLM License, a custom TII license that permits commercial use subject to an acceptable-use policy and an attribution requirement in derivative works, and was published in May 2025.

Pythia 410M Deduped

EleutherAI · 506M · runs from 0.1 GB

Pythia 410M Deduped is a 506M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Deepseek Coder 33B Base

DeepSeek · 33.3B · runs from 14.6 GB

DeepSeek-Coder-33B-Base is DeepSeek's 33.3-billion-parameter base (pretrained, not instruction-tuned) code model, trained from scratch on 2 trillion tokens (87% code, 13% natural language in English and Chinese) using grouped-query attention, project-level code context, and a fill-in-the-blank training objective for project-level completion and infilling. It is intended as a foundation for further fine-tuning or direct code completion and infilling use, not conversational use; the instruction-tuned DeepSeek-Coder-33B-Instruct is built on top of it. At release, the Deepseek Coder family achieved state-of-the-art open-model results on HumanEval, MultiPL-E, MBPP, DS-1000, and APPS. At 33.3 billion parameters, it needs a high-end consumer GPU or a multi-GPU setup once quantized. Context length is 16,384 tokens. Model weights are released under DeepSeek's custom model license, which permits commercial use. It was published in October 2023, alongside 1.3B, 5.7B, and 6.7B siblings and the corresponding instruct checkpoint.

Agnes 3.0 Flash

Agnes-AI · 33.1B · runs from 14.9 GB

Agnes-3.0-Flash Preview is Agnes AI's 33-billion-parameter open-weight multimodal model offering competitive reasoning, coding, and instruction-following at lower hardware cost than flagship-class models; it understands text, images, and video, supports tool calling, and lets developers dial reasoning effort up or down. This open-weight Preview checkpoint is distinct from the larger production/API "Agnes 3.0 Flash" model on third-party leaderboards, which uses a different checkpoint with a 1-million-token context window, so its benchmark results don't transfer to these weights. Architecturally it is a hybrid-attention decoder where three of every four layers use a recurrent gated-delta-rule mechanism instead of standard attention, so only 18 of its 72 layers hold a KV cache that grows with context. At 33 billion dense parameters, it needs a high-end consumer GPU or multi-GPU setup once quantized. Context length is 262,144 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2026.

VisionReasoning

Magistral Small 2506

Mistral AI · 23.6B · runs from 7.2 GB

Magistral Small 2506 is Mistral AI's small, efficient reasoning model, built on the 24-billion-parameter Mistral Small 3.1 and further trained with supervised fine-tuning on Magistral Medium's reasoning traces plus reinforcement learning. It produces long chains of reasoning before answering and supports dozens of languages, making it suited to multi-step problem solving rather than simple chat. The architecture supports up to 128K tokens, but Mistral recommends keeping usage around 40K since output quality can degrade past that point. Released under the Apache 2.0 license, it is compact enough to fit on a single RTX 4090 or a 32 GB RAM Mac once quantized to 4-bit, in line with what a 24-billion-parameter model needs at that precision.

Qwen2.5 Coder 3B

Alibaba · 3.1B · runs from 1.4 GB

Qwen2.5-Coder-3B is Alibaba's 3.1-billion-parameter base (pretrained, not instruction-tuned) code language model, one of six sizes in the Qwen2.5-Coder family built on Qwen2.5-3B. It was pretrained on 5.5 trillion tokens of source code, text-code grounding data, and synthetic data, and is intended as a foundation for further fine-tuning, reinforcement learning, or fill-in-the-middle code-completion tasks rather than direct chat use; Alibaba explicitly advises against using it for conversations as-is. The family's larger 32B model is described as matching GPT-4o-level coding ability, though this 3B checkpoint targets lightweight, resource-constrained deployment. At just over 3 billion parameters, it runs easily on a single modest consumer GPU or even a CPU. Context length is 32,768 tokens. It is released under the Qwen Research License, a non-commercial license restricted to research and evaluation use; commercial use requires a separate license from Alibaba Cloud. It was published in November 2024.

Pythia 6.9B

EleutherAI · 7.0B · runs from 2.1 GB

Pythia 6.9B is a 7.0B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Phi 4 Reasoning Plus

Microsoft · 14.7B · runs from 4.8 GB

Phi 4 Reasoning Plus is a 14.7B-parameter open language model from Microsoft in the Phi 4 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatMathCodeReasoning

Pythia 160M Deduped

EleutherAI · 213M · runs from 0.1 GB

Pythia 160M Deduped is a 213M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Olmo 3 7B Instruct

Allen AI · 7.3B · runs from 3.4 GB

OLMo 3 7B Instruct is an instruction-tuned language model from the Allen Institute for AI, built as part of their Open Language Model initiative. Like all OLMo releases, it comes with fully open training data, code, and intermediate checkpoints, setting a high standard for reproducibility and scientific transparency in the LLM space. At roughly 7 billion parameters, this model delivers competitive performance on instruction following, reasoning, and general knowledge tasks while remaining runnable on consumer GPUs with 8 GB or more of VRAM. It is an excellent choice for users who value open science and want a capable, well-documented model for local chat and assistant applications.

MiniCPM O 2 6

OpenBMB · 8.7B · runs from 3.4 GB

MiniCPM-o 2.6 is OpenBMB's 8.7-billion-parameter omni-modal model, combining a SigLip-400M vision encoder, a Whisper-medium audio encoder, a ChatTTS speech decoder, and a Qwen2.5-7B language backbone trained end to end. Unlike a pure vision-language model, it takes images, video, and audio as input and can generate spoken output too, supporting real-time bilingual speech conversation, voice cloning, and live-streaming understanding. At this size it is practical on a single high-end consumer GPU once quantized. Context length is 32,768 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in January 2025. Its distinguishing feature is the end-to-end omni-modal architecture, using time-division multiplexing that lets it process continuous video and audio streams and reply with synthesized speech, not just text.

GPT Neox 20B

EleutherAI · 20.7B · runs from 6.3 GB

GPT Neox 20B is a 20.7B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

UI TARS 1.5 7B

ByteDance-Seed · 8.3B · runs from 2.7 GB

UI-TARS-1.5-7B is ByteDance's 8.3-billion-parameter vision-language model, built on a Qwen2.5-VL-7B foundation and tuned as a GUI and computer-use agent rather than a chatbot. It reasons through its thoughts before acting, using reinforcement learning to control desktop and browser interfaces, play games, and complete multi-step tasks from screenshots. ByteDance evaluates it against agents like OpenAI's CUA and Claude on benchmarks such as OSWorld and Windows Agent Arena. At this size, local inference is practical on a single mainstream-to-high-end consumer GPU once quantized. It supports a 128,000 token context window, useful for long action histories in agent tasks. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in April 2025, it builds on the original UI-TARS architecture with added inference-time reasoning scaling.

Gemma 4 E4B

Google · 8.0B · runs from 3.9 GB

Gemma 4 E4B is Google DeepMind's second-smallest model in the Gemma 4 family, a dense architecture with roughly 8 billion total parameters, of which Google describes about 4.5 billion as its effective footprint at inference. This is the pretrained base checkpoint rather than an instruction-tuned model, meant for fine-tuning rather than direct chat use. Like the rest of the family it is multimodal — text, image, and natively audio at this size — targeting efficient on-device deployment on laptops and higher-end phones. It runs comfortably on a single consumer GPU, or on-device once quantized. It supports a 131,072 token context window. It carries Google's Gemma 4 license terms, published as Apache 2.0 on Hugging Face with additional Gemma-specific usage terms linked from the card. Published in March 2026, it sits between the E2B on-device model and the larger 12B, 26B-A4B, and 31B tiers.

Qwen3.6 35B A3B DFlash

z-lab · 35B · runs from 15.2 GB

Qwen3.6 35B A3B DFlash is a 35B-parameter open language model from z-lab in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Yi 1.5 9B Chat

01.AI · 8.8B · runs from 4.1 GB

Yi-1.5-9B-Chat is 01.AI's second-generation 9-billion-parameter bilingual (English/Chinese) chat model, continually pretrained from the original Yi series on a further 500 billion high-quality tokens and then fine-tuned on 3 million diverse instruction samples. Compared with the original Yi, Yi-1.5 delivers stronger coding, math, and reasoning while keeping the same language understanding and commonsense reasoning, and 01.AI reports it as the top performer among similarly sized open models on its benchmark suite. At 9B parameters it fits comfortably on a single consumer GPU, or a much smaller card once quantized. Context length is 4,096 tokens; separate 16K- and 32K-context variants of the same model are also available for longer documents. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2024.

Llama 3 3 Nemotron Super 49B V1

NVIDIA · 49.9B · runs from 15.1 GB

Llama 3 3 Nemotron Super 49B V1 is a 49.9B-parameter open language model from NVIDIA in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Meta Llama 3.1 8B Instruct Abliterated

mlabonne · 8.0B · runs from 3.3 GB

Meta Llama 3.1 8B Instruct Abliterated is a 8.0B-parameter open language model from mlabonne in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

EXAONE 3.5 7.8B Instruct

LGAI-EXAONE · 7.8B · runs from 2.7 GB

EXAONE 3.5 7.8B Instruct is a 7.8B-parameter open language model from LGAI-EXAONE in the EXAONE family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 3.1 Tulu 3 8B

Allen AI · 8.0B · runs from 3.3 GB

Llama-3.1-Tulu-3-8B is Allen Institute for AI's fully open instruction-following model, built on Meta's Llama 3.1 8B base through a public pipeline of supervised fine-tuning, direct preference optimization, and a final reinforcement learning stage with verifiable rewards (RLVR). It is tuned for chat as well as harder tasks like math, GSM8K, and instruction-following (IFEval), with all training data, code, and recipes released openly as part of the Tulu 3 project. Its 8B size lets it run on a single consumer GPU. Context length is 131,072 tokens. It is released under the Llama 3.1 Community License Agreement, which restricts use above 700 million monthly active users and imposes Meta's acceptable-use policy. It was published in November 2024, alongside a 70B and later a 405B sibling trained the same way.

Moonlight 16B A3B Instruct

Moonshot AI · 16.0B · runs from 5.1 GB

Moonlight 16B A3B Instruct is a 16.0B-parameter open language model from Moonshot AI in the Moonlight family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

MiniCPM V 4.6 Thinking

OpenBMB · 1.3B · runs from 0.7 GB

MiniCPM-V 4.6 Thinking is OpenBMB's 1.3-billion-parameter vision-language model, a long chain-of-thought reasoning variant of MiniCPM-V 4.6 built on a SigLIP2-400M vision encoder paired with a small Qwen3.5-0.8B language backbone. It generates an explicit reasoning trace before answering, aimed at multimodal reasoning, math, and OCR-heavy document tasks rather than quick captioning, keeping the same edge-friendly, phone-oriented architecture. Its small size lets it run on a single modest consumer GPU. The model supports a 262,144 token context window. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2026. Its distinguishing trait versus base 4.6 is the thinking mode, which trades some latency for better performance on reasoning-heavy visual tasks while reusing the same mixed 4x/16x visual token compression.

Gpt2

OpenAI · 137M · runs from 0.1 GB

GPT-2 is the landmark 2019 language model from OpenAI that helped ignite widespread interest in large-scale text generation. At only 137 million parameters it is tiny by modern standards, but it holds an important place in AI history as the model that was initially deemed too dangerous to release in full. Today GPT-2 runs effortlessly on virtually any hardware, including CPUs, making it ideal for educational purposes, experimentation, and understanding transformer fundamentals. It should not be expected to match the quality of modern instruction-tuned models, but it remains a useful teaching tool and conversation starter.

Apertus 8B Instruct 2509

swiss-ai · 8.1B · runs from 2.8 GB

Apertus 8B Instruct is an open-source instruction-tuned model from Swiss AI, a collaborative research initiative. Built on an 8 billion parameter base, it emphasizes transparency, open data, and European AI sovereignty. For local users, it delivers solid general-purpose chat and instruction-following in a standard 8B footprint that runs well on consumer GPUs with 8 to 10 GB of VRAM, making it a practical choice for those who value open, community-driven model development.

Nemotron Mini 4B Instruct

NVIDIA · 4B · runs from 1.8 GB

Nemotron Mini 4B Instruct is a 4B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.