All LLM Models
Browse 982 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
TinyLlama 1.1B Intermediate Step 1431k 3T
TinyLlama · 1.1B · runs from 0.8 GB
87.9K
192TinyLlama 1.1B Intermediate Step 1431k 3T is a 1.1B-parameter open language model from TinyLlama in the TinyLlama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Starcoder2 3B
BigCode · 3.0B · runs from 1.6 GB
StarCoder2-3B is BigCode's 3-billion-parameter base code model, trained from scratch on 17 programming languages drawn from The Stack v2 using a fill-in-the-middle objective over more than 3 trillion tokens. It is a pretrained checkpoint rather than an instruction-tuned assistant, so it completes and continues code rather than following natural-language commands, and it uses grouped-query attention with a sliding-window mechanism for efficiency. It has a 16,384 token context window and is released under the BigCode OpenRAIL-M license, which carries use-based restrictions rather than being fully permissive. At 3 billion parameters, it runs easily on almost any modern laptop or consumer GPU, even without a high-end card, whether quantized or run at half precision.
EuroLLM 22B Instruct 2512
utter-project · 22.6B · runs from 7.0 GB
EuroLLM 22B Instruct 2512 is a 22.6B-parameter open language model from utter-project. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
H2o Danube3 500M Chat
h2oai · 514M · runs from 0.6 GB
H2o Danube3 500M Chat is a 514M-parameter open language model from h2oai. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 3 Mini 4k Instruct
Microsoft · 3.8B · runs from 2.7 GB
Microsoft Phi 3 Mini 4K Instruct is a 3.8-billion parameter instruction-tuned model from Microsoft Research's Phi 3 generation, with a 4K token context window. The Phi 3 family demonstrated that small models trained on carefully curated, high-quality data can achieve performance competitive with models several times their size. The model runs on consumer GPUs with as little as 4-6GB of VRAM when quantized, making it one of the most accessible capable chat models for local deployment. Released under the MIT license.
Vikhr Nemo 12B Instruct R 21 09 24
Vikhrmodels · 12.2B · runs from 4.8 GB
Vikhr Nemo 12B Instruct R 21 09 24 is a 12.2B-parameter open language model from Vikhrmodels. It supports a context window of up to 1,024,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nanbeige4.1 3B Heretic
heretic-org · 3.9B · runs from 2.1 GB
Nanbeige4.1 3B Heretic is a 3.9B-parameter open language model from heretic-org. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 7B
Alibaba · 7.6B · runs from 3.6 GB
Qwen2.5-7B is Alibaba's 7.6-billion-parameter base pretrained model in the Qwen2.5 series, one step up from the 1.5B checkpoint. Like its smaller sibling it is a raw causal language model rather than an instruction-tuned chat model: the card recommends applying supervised fine-tuning, RLHF, or further pretraining before using it conversationally. It uses a dense transformer with grouped-query attention, and at 7.6B parameters it fits comfortably on a single consumer GPU once quantized, or unquantized on a higher-memory card. Context length is 131,072 tokens, one of the longer windows in the 0.5B-to-72B Qwen2.5 lineup. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in September 2024. Qwen2.5 as a whole brought major gains in coding, math, and structured-data understanding over Qwen2.
INTELLECT 1 Instruct
PrimeIntellect · 10.2B · runs from 3.7 GB
INTELLECT-1-Instruct is the instruction-tuned version of INTELLECT-1, a 10-billion-parameter language model that Prime Intellect describes as the first collaboratively, globally distributed pretraining run of its scale: 1 trillion tokens of English text and code trained across up to 14 concurrent nodes on three continents contributed by 30 independent community participants, using the DiLoCo distributed-training algorithm. Post-training was handled by Arcee AI through supervised fine-tuning, eight rounds of direct preference optimization, and model merging, using the Llama-3 tokenizer and logits distilled from Llama-3.1-405B to improve alignment. At 10B parameters it fits a single consumer GPU once quantized, more comfortably at full precision on a higher-end card. Context length is 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in November 2024, alongside the base INTELLECT-1 model.
Pythia 70M Deduped
EleutherAI · 96M · runs from 0.0 GB
Pythia 70M Deduped is a 96M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Openchat 3.5 0106
OpenChat · 7.2B · runs from 3.6 GB
OpenChat-3.5-0106 is a 7.2-billion-parameter chat model fine-tuned from Mistral-7B-v0.1 using C-RLFT (Conditioned Reinforcement Learning Fine-Tuning), a method that lets the model learn from mixed-quality data by conditioning on data source rather than requiring uniformly high-quality demonstrations. At release, OpenChat billed it as the best-performing open 7B chat model, adding a dedicated coding mode alongside its generalist and math-reasoning mode plus experimental evaluator and feedback capabilities. At 7B parameters it runs on a single consumer GPU, and on modest cards once quantized. Context length is 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in January 2024, an update to the earlier OpenChat-3.5 and OpenChat-3.5-1210 checkpoints.
Tmax 9B
Allen AI · 9.0B · runs from 4.4 GB
Tmax 9B is Allen Institute for AI's terminal-agent model, fine-tuned from Alibaba's Qwen3.5-9B using DPPO, a reinforcement-learning method, on a small curated dataset of terminal-agent trajectories, with the vision head removed so it operates as a text-only, language-model-only checkpoint. It is built specifically to drive command-line and terminal tasks rather than general chat, and clearly outperforms its Qwen3.5-9B base on Terminal-Bench evaluations. Being a single dense 9-billion-parameter model, it runs on a single consumer GPU. Context length is 262,144 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, though Ai2 asks that it be used in line with its Responsible Use Guidelines. It was published in June 2026, as the 9B member of a family of terminal agents that also includes 2B, 4B, and 27B sizes.
Cogito V1 Preview Qwen 32B
deepcogito · 32B · runs from 10.4 GB
Cogito V1 Preview Qwen 32B is a 32B-parameter open language model from deepcogito in the Qwen family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2 0.5B
Alibaba · 494M · runs from 0.5 GB
Qwen2-0.5B is Alibaba's roughly 494-million-parameter dense base language model, the smallest member of the Qwen2 family that scales up to 72B and includes one Mixture-of-Experts variant. This is a pretrained base model, not an instruction-tuned chat model; Alibaba explicitly recommends applying supervised fine-tuning, RLHF, or further pretraining before using it for open-ended generation. Given its size, it runs easily on nearly any GPU and even on CPUs. Its configuration specifies a context window of up to 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2024. It was superseded within the same year by Qwen2.5-0.5B, part of Alibaba's fast release cadence for its small-model line.
Hermes 4.3 36B
Nous Research · 36.2B · runs from 10.5 GB
Hermes 4.3 36B is a 36.2B-parameter open language model from Nous Research in the Hermes family. It supports a context window of up to 524,288 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.6 27B DFlash
z-lab · 27B · runs from 11.8 GB
Qwen3.6 27B DFlash is a 27B-parameter open language model from z-lab in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 4 Reasoning
Microsoft · 14.7B · runs from 4.8 GB
Phi-4-reasoning is Microsoft's reasoning-focused fine-tune of Phi-4, a 14-billion-parameter dense decoder-only transformer, produced through supervised fine-tuning on chain-of-thought traces followed by reinforcement learning, concentrated on math, science, and coding tasks plus safety alignment data. The fine-tuning set combined synthetic prompts with high-quality filtered public web data, and responses are structured in two parts: a reasoning trace followed by a final summarized answer. Microsoft designed it for memory- and compute-constrained, latency-bound scenarios rather than frontier-scale serving, and at 14B parameters it can run on a single high-end consumer GPU once quantized. Context length is 32,768 tokens. It is released under the MIT license, permitting unrestricted commercial and research use, and was published in April 2025.
Qwen3 0.6B Base
Alibaba · 596M · runs from 0.7 GB
Qwen3 0.6B Base is the smallest pretrained foundation model in Alibaba Cloud's Qwen 3 family, with approximately 600 million parameters. As a base model, it is not tuned for chat or instructions and is intended for fine-tuning, research, and experimentation. Its minimal size makes it suitable for rapid prototyping and resource-constrained training experiments. The model runs on virtually any hardware, including CPU-only setups. It is useful for educational purposes, architecture exploration, and as a compact foundation for task-specific fine-tuning where model size is a primary constraint. Released under the Apache 2.0 license.
GPT OSS 20B RichardErkhov Heresy
MuXodious · 21.5B · runs from 6.3 GB
GPT OSS 20B RichardErkhov Heresy is a 21.5B-parameter open language model from MuXodious in the GPT-OSS family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Olmo 3 7B Think
Allen AI · 7.3B · runs from 3.4 GB
Olmo 3 7B Think is a 7.3B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 65,536 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mellum2 12B A2.5B Thinking
JetBrains · 12.1B · runs from 4.0 GB
Mellum2-12B-A2.5B-Thinking is JetBrains' reasoning-augmented assistant model, a mixture-of-experts model with 64 experts and 8 active per token, roughly 2.4 billion active out of about 12.1 billion total parameters. It is produced from the Mellum2-12B-A2.5B-Base checkpoint through supervised fine-tuning followed by reinforcement learning with verifiable rewards on a harder data mix that includes long-form math, and it emits its reasoning inside dedicated think blocks before the final answer. A separate Instruct sibling in the same family skips the reasoning step for lower-latency, direct answers. At this size it fits on a single high-end consumer GPU once quantized. Context length is 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2026.
OpenELM 1 1B Instruct
Apple · 1.1B · runs from 0.5 GB
OpenELM-1.1B-Instruct is Apple's 1.08-billion-parameter instruction-tuned language model, using a layer-wise scaling strategy that varies parameter allocation across layers for better accuracy per parameter. It's a small, general-purpose chat model for on-device and research use. Apple released it alongside 270M, 450M, and 3B siblings and, unusually for the company, published the full pretraining and fine-tuning pipeline for reproducibility. Its size makes it easy to run on a single consumer GPU or a modern laptop CPU. Context length is limited to 2,048 tokens, reflecting its 2024-era pretraining. It is released under Apple's Machine Learning Research license, permitting use and modification for research rather than the broad commercial rights of Apache or MIT. Published in April 2024, it trained on roughly 1.8 trillion tokens from RefinedWeb, deduplicated PILE, RedPajama, and Dolma.
Gemma 2B
Google · 2.5B · runs from 1.2 GB
Gemma 2B is a 2.5B-parameter open language model from Google in the Gemma 2 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Pythia 410M
EleutherAI · 506M · runs from 0.2 GB
Pythia 410M is a 506M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 8B Base
Alibaba · 8.2B · runs from 4.1 GB
Qwen3 8B Base is an 8.2-billion parameter pretrained foundation model from Alibaba Cloud's Qwen 3 series. As a base model, it is not instruction-tuned and is intended for fine-tuning, research, and as a starting point for custom downstream applications. It was trained on a large multilingual corpus with improved data quality and training methodology compared to the Qwen 2.5 generation. The model runs efficiently on consumer GPUs with 8GB or more of VRAM and serves as the foundation for the Qwen3 8B instruction-tuned variant and community fine-tunes. It is a strong choice for practitioners building specialized models through further training. Released under the Apache 2.0 license.
Qwen2 1.5B Instruct
Alibaba · 1.5B · runs from 0.8 GB
Qwen2 1.5B Instruct is a 1.5B-parameter open language model from Alibaba in the Qwen 2 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron 3 Nano 4B BF16
NVIDIA · 4.0B · runs from 2.2 GB
NVIDIA Nemotron 3 Nano 4B BF16 is a 4.0B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 4 E2B
Google · 5.1B · runs from 2.5 GB
Gemma 4 E2B is Google DeepMind's smallest model in the Gemma 4 family, a dense architecture with roughly 5.1 billion total parameters, of which Google describes about 2.3 billion as its effective footprint at inference. This is the pretrained base checkpoint, not an instruction-tuned chat model, meant as a starting point for fine-tuning. The Gemma 4 family is multimodal — text, image, and at this size natively audio — and E2B targets efficient on-device execution on phones and laptops. It is easy to run locally, even on modest hardware once quantized. It supports a 131,072 token context window. It carries Google's Gemma 4 license terms, published as Apache 2.0 on Hugging Face with additional Gemma-specific usage terms linked from the card. Published in March 2026, it is the smallest of five sizes in the Gemma 4 lineup, aimed at mobile and edge deployment.
NeuralDaredevil 8B Abliterated
mlabonne · 8.0B · runs from 4.0 GB
NeuralDaredevil 8B Abliterated is a 8.0B-parameter open language model from mlabonne. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 14B Base
Alibaba · 14.8B · runs from 4.7 GB
Qwen3 14B Base is a 14.8B-parameter open language model from Alibaba in the Qwen 3 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.