All LLM Models
Browse 1242 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
SmolLM2 135M
Hugging Face · 135M · runs from 0.4 GB
1.8M
235SmolLM2 135M is one of the smallest capable language models available, developed by Hugging Face as part of their SmolLM2 family. With just 135 million parameters, it requires virtually no VRAM and can run on almost any hardware, making it an excellent starting point for researchers experimenting with language model behavior, fine-tuning workflows, or edge deployment scenarios. Despite its tiny footprint, SmolLM2 135M benefits from improved training data and techniques compared to its first-generation predecessor. It is best suited for lightweight text generation tasks, prototyping, and educational purposes rather than production-grade applications.
Llama 68M
JackFram · 68M · runs from 0.0 GB
Llama 68M is a 68M-parameter open language model from JackFram in the Llama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 1 5
Microsoft · 1.4B · runs from 0.7 GB
Phi 1 5 is a 1.4B-parameter open language model from Microsoft in the Phi family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Manaca 1B Base
menezesbruno · 1.7B · runs from 1.1 GB
Manaca 1B Base is a 1.7B-parameter open language model from menezesbruno. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen 7B
Alibaba · 7.7B · runs from 3.6 GB
Qwen-7B is Alibaba's original 7.7-billion-parameter base language model from the first Qwen series, pretrained (not instruction-tuned) on over 2.4 trillion tokens of Chinese, English, multilingual, code, and math text, with a roughly 150,000-token vocabulary aimed at multilingual friendliness. Alibaba positions it as a foundation for further fine-tuning; the aligned chat assistant Qwen-7B-Chat is a separate release built on top of it. Its native training context is 8,192 tokens, and the card documents NTK-aware interpolation, LogN attention scaling, and windowed attention to extrapolate usable context out to 32,768 tokens. At under 8 billion parameters, it fits on a single consumer GPU. Context length is 8,192 tokens natively, extendable to 32,768 tokens with the NTK, LogN, and window-attention techniques above. It is released under the Tongyi Qianwen License Agreement, which allows academic use freely but requires applying for a commercial license for commercial deployment. It was published in August 2023, an early entry in a Qwen lineage now several generations ahead.
Salamandra 7B Instruct
BSC-LT · 7.8B · runs from 3.8 GB
Salamandra 7B Instruct is a 7.8-billion-parameter multilingual model developed by the Barcelona Supercomputing Center (BSC-LT) as part of a European initiative to build high-quality open language models. It has particular strength in Iberian languages including Spanish, Catalan, Portuguese, and Basque, while also supporting English and other major European languages. This model is an excellent choice for users who need strong performance in Spanish or other Iberian languages that are often underserved by mainstream LLMs. Running it locally ensures data privacy for sensitive multilingual workflows, and at 7B parameters it fits comfortably on a single consumer GPU with 8 GB or more of VRAM.
Qwen2.5 Math 7B
Alibaba · 7.6B · runs from 3.6 GB
Qwen2.5 Math 7B is a 7.6B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Molmo2 8B
Allen AI · 8.7B · runs from 4.3 GB
Molmo2 8B is an 8.7-billion-parameter vision-language model from the Allen Institute for AI (Ai2), built on a Qwen3-8B backbone paired with a SigLIP2 vision encoder. It is tuned for image, video, and multi-image understanding, with particular strength in pointing, counting, and tracking objects across frames rather than just describing them. Its size suits local deployment on a mainstream-to-high-end consumer GPU once quantized, a step down from the 72B Molmo model it succeeds. The model supports a 36,864 token context window, adequate for multi-image prompts and moderate video rather than long documents. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in December 2025, Molmo2 8B is notable for pixel-level grounding and for tracking objects over time in video.
Sarvam 30B
sarvamai · 32.2B · runs from 14 GB
Sarvam 30B is a 32.2B-parameter open language model from sarvamai. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen1.5 7B
Alibaba · 7.7B · runs from 4.7 GB
Qwen1.5-7B is Alibaba's dense 7.7-billion-parameter base language model, part of the Qwen1.5 series that served as a beta step between the original Qwen and Qwen2. It is a decoder-only Transformer using SwiGLU activation, released as a pretrained checkpoint rather than a chat model; its card explicitly recommends applying further post-training such as SFT or RLHF rather than prompting it directly for text generation. It offers a stable 32K token context window and is distributed under Alibaba's Tongyi Qianwen license, a custom license rather than a fully open one. At 7.7 billion parameters, it runs comfortably on an 8-12 GB consumer GPU once quantized to 4-bit, making it an accessible option for local fine-tuning experiments.
PowerMoE 3B
ibm-research · 3.4B · runs from 1.7 GB
PowerMoE-3B is IBM Research's 3.4-billion-parameter sparse Mixture-of-Experts language model, built for general text generation, reasoning, and code rather than a specific vision or audio modality. It routes each token to 8 of 40 experts, activating roughly 800 million parameters per token according to IBM, keeping decoding fast even though the full set of experts still has to be held in memory. At this size it fits comfortably on a single mainstream consumer GPU once quantized. Context length is limited to 4,096 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in August 2024. Its defining feature is the Power learning-rate scheduler IBM used to train it, which the team reports lets an 800-million-active-parameter model match dense models with roughly twice as many active parameters.
Seed OSS 36B Base
ByteDance-Seed · 36.2B · runs from 15.9 GB
Seed-OSS-36B-Base is ByteDance's 36-billion-parameter open-source base language model, a dense transformer using GQA attention, RMSNorm, and SwiGLU, designed as a foundation for long-context, reasoning, and agentic downstream use. It is a pretrained model, not instruction-tuned, though this particular checkpoint's pretraining mix includes synthetic instruction data (ByteDance separately released a Base-woSyn variant trained without it, for researchers who want a foundation model unaffected by synthetic instruction data). Despite training on only 12 trillion tokens, it scores competitively against larger open pretrained models on knowledge, reasoning, and math benchmarks. At 36B parameters it needs a multi-GPU workstation or heavy quantization to run locally. Context length is 524,288 tokens, trained natively rather than extended after the fact. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in August 2025 alongside the instruction-tuned Seed-OSS-36B-Instruct.
LocoTrainer 4B
LocoreMind · 4.0B · runs from 2.2 GB
LocoTrainer 4B is a 4.0B-parameter open language model from LocoreMind. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Prometheus 7B V2.0
prometheus-eval · 7.2B · runs from 3.6 GB
Prometheus 7B V2.0 is a specialized judge model trained by prometheus-eval to evaluate the quality of outputs from other language models. At 7.2 billion parameters, it is designed to score and critique LLM responses against custom rubrics, making it a valuable tool for automated evaluation pipelines and benchmarking. Unlike general-purpose chat models, Prometheus is purpose-built for assessment tasks. It can provide structured feedback on dimensions like helpfulness, accuracy, and coherence. Useful for researchers, developers building LLM applications, and anyone who needs consistent automated evaluation without relying on paid API calls to frontier models. Runs comfortably on most modern GPUs with 8 GB or more of VRAM.
ReaderLM v2
jinaai · 1.5B · runs from 1.0 GB
ReaderLM v2 is a 1.5B-parameter open language model from jinaai. It supports a context window of up to 512,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Pebble 10M Chat
basically-ai · 11M · runs from 0.0 GB
Pebble 10M Chat is a 11M-parameter open language model from basically-ai. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 3.1 70B LatamGPT SFT 1.0
latam-gpt · 70.6B · runs from 24.8 GB
Llama 3.1 70B LatamGPT SFT 1.0 is a 70.6B-parameter open language model from latam-gpt in the Llama 3 family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
CodeQwen1.5 7B
Alibaba · 7.3B · runs from 3.5 GB
CodeQwen1.5-7B is Alibaba's 7.3-billion-parameter code-specialized base language model, built on the Qwen1.5 architecture and pretrained on 3 trillion tokens of code data covering 92 programming languages. It uses group-query attention for efficient inference and is a raw pretrained model, not tuned for chat; Alibaba advises against using it directly for conversation and instead for fine-tuning, code infilling, and code generation. The card highlights strong text-to-SQL and bug-fixing performance for its size. At 7B parameters it fits comfortably on a single consumer GPU, even less once quantized. Context length is 65,536 tokens. It is released under a custom Tongyi Qianwen Research License, free for academic research, with commercial use requiring direct contact with Alibaba. It was published in April 2024.
Qwen1.5 7B Chat
Alibaba · 7.7B · runs from 4.7 GB
Qwen1.5 7B Chat is a 7.7B-parameter open language model from Alibaba in the Qwen family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma 3 4B T1 IT
twinkle-ai · 3.9B · runs from 2.3 GB
Gemma 3 4B T1 IT is a 3.9B-parameter open language model from twinkle-ai in the Gemma 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
SmolLM2 360M
Hugging Face · 362M · runs from 0.5 GB
SmolLM2 360M is a 362M-parameter open language model from Hugging Face in the SmolLM family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Amber
LLM360 · 6.7B · runs from 3.2 GB
Amber is a 6.7 billion parameter model from LLM360, an initiative dedicated to full transparency in large language model training. Every aspect of Amber's creation has been publicly documented and released, including the complete training data, all intermediate checkpoints, training code, and evaluation results. This level of openness makes Amber uniquely valuable for researchers studying training dynamics, data influence, and model behavior at scale. For local deployment, it offers solid general-purpose text generation at a size that fits comfortably on mid-range consumer GPUs, though users primarily seeking chat performance may prefer models specifically tuned for instruction following.
Mellum 4B Base
JetBrains · 4.0B · runs from 2.8 GB
Mellum 4B Base is a 4.0B-parameter open language model from JetBrains in the Mellum family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 160M
JackFram · 162M · runs from 0.4 GB
Llama 160M is a 162M-parameter open language model from JackFram in the Llama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Olmo 3 32B Think SFT
Allen AI · 32.2B · runs from 14.5 GB
Olmo 3 32B Think SFT is a 32.2B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 65,536 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Distilgpt2
distilbert · 88M · runs from 0.0 GB
DistilGPT-2 is a distilled version of OpenAI's GPT-2 model, compressed to just 88 million parameters while retaining much of the original model's text generation ability. Created using knowledge distillation techniques, it offers significantly faster inference than the full GPT-2 with only a modest reduction in output quality. This model is one of the lightest autoregressive language models available and can run on virtually any hardware, including CPUs. It is a practical choice for educational projects, quick prototyping, and applications where inference speed and minimal resource usage are more important than state-of-the-art generation quality.
Supra2 100M Instruct
SupraLabs · 101M · runs from 0.4 GB
Supra2 100M Instruct is a 101M-parameter open language model from SupraLabs. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gpt2 Mini
erwanf · 39M · runs from 0.0 GB
Gpt2 Mini is a 39M-parameter open language model from erwanf. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Falcon 7B
TII UAE · 7.2B · runs from 3.4 GB
Falcon 7B was one of the first truly competitive open-source large language models, released in mid-2023 by the Technology Innovation Institute in Abu Dhabi. Trained on the massive RefinedWeb dataset, it demonstrated that carefully curated web data could rival models trained on more traditionally assembled corpora. At 7 billion parameters, Falcon 7B helped establish the 7B class as the sweet spot for local inference, offering genuine language understanding on consumer GPUs with as little as 6 GB of VRAM.
Bloomz 560M
BigScience · 559M · runs from 0.3 GB
Bloomz 560M is a 559M-parameter open language model from BigScience. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.