All LLM Models
Browse 1475 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Llama 3.1 405B Instruct
Meta · 405.9B · runs from 189.7 GB
12.7K
602Llama 3.1 405B Instruct is Meta's largest Llama 3.1 model, a 405.9-billion-parameter dense (non-mixture-of-experts) Transformer instruction-tuned for chat, tool use, and multilingual tasks across 8 languages. At release it was the first openly available model benchmarked as competitive with leading proprietary models on general knowledge, math, and reasoning. Running it at full precision requires a multi-GPU server; even quantized, it needs a substantial multi-GPU workstation rather than a single consumer card. Context length is 131,072 tokens (128K), a large jump from the 8,192-token window of the original Llama 3. It is released under the Llama 3.1 Community License, a custom license that requires organizations with more than 700 million monthly active users to obtain a separate license from Meta and imposes an acceptable-use policy on prohibited applications. It was published in July 2024, alongside the smaller 8B and 70B Llama 3.1 models.
Qwen3.8 27B Abliterated MTPLX Optimized Speed
PocketAiHub · 26.9B · runs from 54.5 GB
Qwen3.8 27B Abliterated MTPLX Optimized Speed is a 26.9B-parameter open language model from PocketAiHub in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
K2 Horizon 3.7B
IFM · 5.1B · runs from 10.6 GB
K2 Horizon 3.7B is a 5.1B-parameter open language model from IFM. It supports a context window of up to 524,288 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nex N2.5 Max
nex-agi · 1600.8B · runs from 3201.9 GB
Nex N2.5 Max is a 1600.8B-parameter open language model from nex-agi. It supports a context window of up to 1,048,576 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Lfm2.5 2.6B Fable5 Coding Agent Heretic
saidutta69 · 2.7B · runs from 1.6 GB
Lfm2.5 2.6B Fable5 Coding Agent Heretic is a 2.7B-parameter open language model from saidutta69 in the LFM2.5 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
TinyStories 33M
roneneldan · 33M · runs from 0.1 GB
TinyStories 33M is a 33M-parameter open language model from roneneldan. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 3n E4B IT Litert Lm
Google · 4B · runs from 1.9 GB
Gemma 3n E4B IT Litert Lm is a 4B-parameter open language model from Google in the Gemma 3 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
SOLAR 10.7B v1.0
Upstage · 10.7B · runs from 5.3 GB
SOLAR 10.7B v1.0 is a 10.7B-parameter open language model from Upstage in the Solar family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwable 9B Claude Fable 5
empero-ai · 9.4B · runs from 19.4 GB
Qwable 9B Claude Fable 5 is a 9.4B-parameter open language model from empero-ai. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.8 2B
empero-ai · 2.3B · runs from 1.4 GB
Qwen3.8 2B is a 2.3B-parameter open language model from empero-ai in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NuExtract 1.5 Tiny
numind · 494M · runs from 0.5 GB
NuExtract 1.5 Tiny is a 494M-parameter open language model from numind. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 3 Medium 4k Instruct
Microsoft · 14.0B · runs from 6.7 GB
Phi 3 Medium 4k Instruct is a 14.0B-parameter open language model from Microsoft in the Phi 3 family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Jack 3.8 27B Coder 16GB VRAM
JackAgentLead · 27B · runs from 59.4 GB
Jack 3.8 27B Coder 16GB VRAM is a 27B-parameter open language model from JackAgentLead. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Kimi K2 Base
Moonshot AI · 1026.5B · runs from 440.1 GB
Kimi K2 Base is a 1026.5B-parameter open language model from Moonshot AI in the Kimi K2 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Internlm2 5 20B Chat
InternLM · 19.9B · runs from 9.1 GB
Internlm2 5 20B Chat is a 19.9B-parameter open language model from InternLM in the InternLM family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
C4ai Command R Plus
Cohere · 103.8B · runs from 48.5 GB
C4ai Command R Plus is a 103.8B-parameter open language model from Cohere in the Command R family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
ThinkingCap Qwen3.6 27B
bottlecapai · 27.4B · runs from 12.4 GB
ThinkingCap Qwen3.6 27B is a 27.4B-parameter open language model from bottlecapai in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Rwkv 4 169M Pile
RWKV · 169M · runs from 0.1 GB
Rwkv 4 169M Pile is a 169M-parameter open language model from RWKV. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 1
Microsoft · 1.4B · runs from 0.7 GB
Phi 1 is a 1.4B-parameter open language model from Microsoft in the Phi family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
WizardLM 2 8x22B
alpindale · 140.6B · runs from 60.5 GB
WizardLM-2 8x22B is a 140.6-billion-parameter mixture-of-experts chat model built by Microsoft's WizardLM team on top of Mixtral-8x22B-v0.1, tuned for complex chat, multilingual conversation, reasoning, and agent tasks. Microsoft briefly published it in April 2024 alongside smaller 70B and 7B siblings, then withdrew the official release within hours after disclosing that a required toxicity test had not been completed; this repository is a community mirror uploaded from the brief public window before the takedown. As an 8x22B mixture-of-experts model, it needs a multi-GPU workstation to run even quantized. Context length is 65,536 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, though as an unofficial mirror of a withdrawn release its provenance is less certain than an official checkpoint. It was published in April 2024.
Llama 3 ELYZA JP 8B
elyza · 8.0B · runs from 4.0 GB
Llama 3 ELYZA JP 8B is a 8.0B-parameter open language model from elyza in the Llama 3 family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
PapersRAG 1.5B
metaresearch · 1.5B · runs from 1.0 GB
PapersRAG 1.5B is a 1.5B-parameter open language model from metaresearch. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
DeepSeek V4 Flash DSpark Abliterated Uncensored
drowzeys · 165.3B · runs from 77.3 GB
DeepSeek V4 Flash DSpark Abliterated Uncensored is a 165.3B-parameter open language model from drowzeys in the DeepSeek V4 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
T5gemma 2B 2B Ul2
Google · 5.6B · runs from 2.6 GB
T5gemma 2B 2B Ul2 is a 5.6B-parameter open language model from Google in the Gemma 2 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Ouro 1.4B Thinking
ByteDance · 1.4B · runs from 3.6 GB
Ouro 1.4B Thinking is a 1.4B-parameter open language model from ByteDance. It supports a context window of up to 65,536 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.8 2B Distill
empero-ai · 2.3B · runs from 1.4 GB
Qwen3.8 2B Distill is a 2.3B-parameter open language model from empero-ai in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Sarvam 1
sarvamai · 2.5B · runs from 1.6 GB
Sarvam 1 is a 2.5B-parameter open language model from sarvamai. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Trinity Large Thinking
Arcee AI · 398.6B · runs from 797.8 GB
Trinity Large Thinking is a 398.6B-parameter open language model from Arcee AI. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Olmo 3.1 32B Think
Allen AI · 32.2B · runs from 65.3 GB
Olmo 3.1 32B Think is a 32.2B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 65,536 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Granite 3.1 2B Instruct
IBM · 2.5B · runs from 1.5 GB
Granite 3.1 2B Instruct is IBM's 2.5-billion-parameter dense instruction-tuned model, fine-tuned from Granite-3.1-2B-Base on permissively licensed open instruction datasets plus internally generated synthetic data aimed at long-context problems. It targets business-assistant work such as summarization, classification, extraction, question answering, retrieval-augmented generation, code tasks and function calling, and supports twelve languages including English, German, Spanish, French, Japanese, Arabic and Chinese. At this size it runs on almost any consumer GPU, and on a laptop CPU once quantized. Context length is 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2024. IBM has since superseded it with Granite 3.3 2B Instruct, which keeps the same size class.