All LLM Models

Browse 1242 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Qwen2.5 1.5B

Alibaba · 1.5B · runs from 1 GB

Qwen2.5-1.5B is Alibaba's 1.5-billion-parameter entry in the Qwen2.5 family, released as a base pretrained language model rather than an instruction-tuned chat model. Qwen2.5 improved knowledge, coding, and math capabilities over its predecessor through an expanded pretraining corpus, but this checkpoint is the raw base: the model card explicitly discourages using it directly for conversation and recommends fine-tuning (SFT, RLHF, or continued pretraining) first. At this size, it runs comfortably on almost any modern GPU, or even a CPU, without quantization. The model supports a 131,072 token context window and is released under the Apache 2.0 license, allowing unrestricted commercial and research use. It was published in September 2024 alongside the rest of the Qwen2.5 line, which spans from 0.5B to 72B parameters and adds multilingual support for over 29 languages.

HarmBench Llama 2 13B Cls

cais · 13.0B · runs from 5.6 GB

HarmBench Llama 2 13B Cls is a 13.0B-parameter open language model from cais in the Llama 2 family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Falcon 11B

TII UAE · 11.1B · runs from 5.0 GB

Falcon2-11B is TII's base pretrained causal decoder model, not instruction-tuned, with 11 billion parameters trained on over 5,000 billion tokens of the RefinedWeb dataset plus curated corpora spanning English and nine other European languages. Training proceeded in stages that grew the context window from 2,048 to 8,192 tokens before a final high-quality data stage. TII recommends further fine-tuning before production use. At 11 billion parameters it runs on a single consumer GPU, especially once quantized. Context length is 8,192 tokens. It is released under the TII Falcon License 2.0, a custom Apache 2.0-based license that adds an acceptable-use policy for responsible use of the model. It was published in May 2024, and has since been superseded by the Falcon3 series.

Starcoder2 7B

BigCode · 7.2B · runs from 3.5 GB

Starcoder2 7B is a 7.2B-parameter open language model from BigCode in the StarCoder family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.5 9B Claude 4.6 Opus Reasoning Distilled

Jackrong · 9.7B · runs from 4.7 GB

Qwen3.5 9B Claude 4.6 Opus Reasoning Distilled is a 9.7B-parameter open language model from Jackrong in the Qwen 3.5 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.5 35B A3B Base

Alibaba · 36.0B · runs from 10.3 GB

Qwen3.5-35B-A3B-Base is the mixture-of-experts member of Alibaba's Qwen3.5 base family, pairing a 256-expert MoE layer (8 routed plus 1 shared expert per token) with the same Gated DeltaNet and gated-attention hybrid backbone used across the line. It totals roughly 36 billion parameters but activates only about 3 billion per token (the "A3B" in its name), so decoding stays fast even though the full expert set must stay resident in memory. Like its dense siblings it is pretrain-only, for fine-tuning and research, not direct chat; unlike them, its card lists only a pretraining stage, with no post-training pass. It needs a high-end consumer GPU once quantized. Context length is 262,144 tokens natively, extensible to 1,010,000 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2026, alongside four smaller dense Qwen3.5-Base models from 0.8B to 9B parameters.

SmolLM 360M

Hugging Face · 362M · runs from 0.5 GB

SmolLM-360M is Hugging Face's 360-million-parameter base language model, one of three sizes (135M, 360M, and 1.7B) in the original SmolLM family. It is a pretrained model, not instruction-tuned or chat-formatted, trained on 600 billion tokens from the curated Cosmo-Corpus dataset: Cosmopedia v2 synthetic textbooks and stories, Python-Edu educational code samples, and FineWeb-Edu deduplicated educational web text. At this size it is intended as a lightweight research and fine-tuning base rather than a ready chat assistant, and it runs comfortably on a CPU or any consumer GPU. Context length is 2,048 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in July 2024.

InternVL3 1B

OpenGVLab · 938M · runs from 0.3 GB

InternVL3-1B is OpenGVLab's smallest vision-language model in the InternVL3 series, combining an InternViT-300M-448px-V2.5 vision encoder with a Qwen2.5-0.5B language backbone through an MLP projector. Unlike earlier InternVL generations, it uses Native Multimodal Pre-Training, interleaving image-text and video-text data with text-only corpora in a single pretraining stage rather than adapting a finished text model, and adds Variable Visual Position Encoding for better long-context handling. The series extends its capabilities to tool use, GUI agents, industrial image analysis, and 3D vision perception. At under 1 billion parameters, it is small enough to run on a modest consumer GPU or even a CPU. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in April 2025, as the smallest member of a family that scales up to InternVL3-78B.

Carnice V1 9B Hermes Agent Stage2 Merged

kai-os · 9.0B · runs from 4.4 GB

Carnice V1 9B Hermes Agent Stage2 Merged is a 9.0B-parameter open language model from kai-os in the Hermes family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatFunctionsReasoning

EuroLLM 1.7B Instruct

utter-project · 1.7B · runs from 1.2 GB

EuroLLM 1.7B Instruct is a 1.7B-parameter open language model from utter-project. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Dobby Mini Unhinged Llama 3.1 8B

SentientAGI · 8.0B · runs from 2.8 GB

Dobby Mini Unhinged Llama 3.1 8B is a 8.0B-parameter open language model from SentientAGI in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Athene v2 Chat

Nexusflow · 72.7B · runs from 21.0 GB

Athene-V2-Chat-72B is Nexusflow's 72-billion-parameter chat model, fine-tuned from Qwen2.5-72B-Instruct through reinforcement learning from human feedback (RLHF) to specialize in chat, math, and coding. On Chatbot Arena the card reports it beating GPT-4o-0513 in the hard-prompts and math categories and matching it on coding, instruction-following, and multi-turn chat. A sister model, Athene-V2-Agent-72B, is fine-tuned separately for function calling and agentic tasks. At 72B parameters, it needs a multi-GPU workstation or server to run, even quantized. Context length is 32,768 tokens. It is released under the Nexusflow Research License, a custom license restricted to personal, non-profit, non-commercial use, with commercial use requiring separate permission from Nexusflow, and was published in November 2024.

Qwen2 7B

Alibaba · 7.6B · runs from 3.6 GB

Qwen2 7B is a 7.6B-parameter open language model from Alibaba in the Qwen 2 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Meta Llama 3 70B Instruct

Meta · 70.6B · runs from 23.3 GB

Meta Llama 3 70B Instruct is a 70.6-billion parameter instruction-tuned model from Meta's Llama 3 release. It is fine-tuned for dialogue, coding assistance, and complex reasoning tasks using supervised fine-tuning and RLHF. At the time of release, it was among the most capable openly available models. The model supports an 8K token context window and requires substantial VRAM for local inference, typically needing multi-GPU setups or high-VRAM professional GPUs. It has been widely adopted for local deployment in quantized formats. Released under the Meta Llama 3 Community License.

Qwen2 72B Instruct

Alibaba · 72.7B · runs from 21.0 GB

Qwen2-72B-Instruct is the instruction-tuned, chat-ready 72.7-billion-parameter model from Alibaba's Qwen2 series, built on a Transformer architecture with SwiGLU activation, QKV attention bias, and grouped-query attention, and trained with supervised fine-tuning plus direct preference optimization. It generally surpassed the prior Qwen1.5 line and competed with proprietary models on language understanding, generation, multilingual tasks, coding, and math benchmarks at release, though it has since been superseded by Qwen2.5-72B-Instruct. At 72.7 billion parameters, it needs a multi-GPU workstation to run in full precision. Context length is natively 32,768 tokens, extendable to 131,072 tokens using YaRN scaling, as documented on the model card. It is released under Alibaba's Tongyi Qianwen custom license, which permits commercial use but requires a separate license once a deployment passes 100 million monthly active users. It was published in May 2024.

Eurus 2 7B PRIME

PRIME-RL · 7.6B · runs from 3.0 GB

Eurus-2-7B-PRIME is a 7.6-billion-parameter reasoning model from the PRIME-RL project, built on Qwen2.5-Math-7B-Base and trained with PRIME (Process Reinforcement through Implicit Rewards), an open-source online reinforcement-learning method that scores intermediate reasoning steps rather than only final answers. It starts from the Eurus-2-7B-SFT checkpoint and is trained further on the Eurus-2-RL-Data set, focused on math and coding problem-solving. The card reports a 16.7% average improvement over the SFT starting point, with over 20% gains on AMC and AIME competition math, enough to surpass the larger Qwen2.5-Math-7B-Instruct on several reasoning benchmarks. At this size it fits on a single consumer GPU. Context length is 4,096 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2024.

InternVL3 8B

OpenGVLab · 7.9B · runs from 2.4 GB

InternVL3-8B is OpenGVLab's roughly 7.9-billion-parameter vision-language model, pairing an InternViT-300M vision encoder with a Qwen2.5-7B language backbone in a ViT-MLP-LLM architecture. It handles general image and video understanding and document analysis, extending into tool use, GUI agent tasks, and 3D scene perception beyond typical captioning. It is comfortably runnable on a single mainstream-to-high-end consumer GPU once quantized. Its language backbone supports a 32,768 token context window. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2025 as part of a 1B-to-78B InternVL3 family sharing the same vision encoder. Its key change versus InternVL2.5 is Native Multimodal Pre-Training, which trains vision and language jointly from the start instead of adapting a language-only model afterward.

LFM2.5 1.2B JP 202606

Liquid AI · 1.2B · runs from 0.9 GB

LFM2.5 1.2B JP 202606 is a 1.2B-parameter open language model from Liquid AI in the LFM2.5 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Saiga Llama3 8B

IlyaGusev · 8.0B · runs from 4.0 GB

Saiga Llama3 8B is a 8.0B-parameter open language model from IlyaGusev in the Llama 3 family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Yi 9B

01.AI · 8.8B · runs from 4.1 GB

Yi-9B is 01.AI's 8.8-billion-parameter base language model, continuously pretrained from Yi-6B on an additional 0.8 trillion tokens as part of the bilingual English/Chinese Yi series (trained on 3 trillion tokens overall). It is a pretrained model, not instruction-tuned; a separate long-context Yi-9B-200K variant exists for extended-context use. The card reports it as the strongest model in its size class among Mistral-7B, SOLAR-10.7B, Gemma-7B, and DeepSeek-Coder-7B-Base, particularly in code, math, common-sense reasoning, and reading comprehension. It fits on a single consumer GPU. Context length is 4,096 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in March 2024.

Pythia 1B

EleutherAI · 1.1B · runs from 0.5 GB

Pythia 1B is a 1.1B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gpt2 Medium

OpenAI · 380M · runs from 0.2 GB

GPT-2 Medium scales the original GPT-2 architecture to 380 million parameters, offering noticeably improved text generation quality over the base 137M variant while remaining extremely lightweight by current standards. It supports the same autoregressive language modeling tasks as its smaller and larger siblings. Like all GPT-2 variants, it runs comfortably on virtually any modern hardware including CPU-only setups, making it an accessible option for learning, prototyping, and lightweight text generation experiments without needing a dedicated GPU.

Granite 4.0 Tiny Preview

IBM · 6.7B · runs from 2.7 GB

Granite 4.0 Tiny Preview is a 6.7B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Granite Guardian 3.3 8B

IBM · 8.2B · runs from 2.9 GB

Granite Guardian 3.3 8B is a 8.2B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

PaddleOCR VL 1.6

PaddlePaddle · 959M · runs from 0.6 GB

PaddleOCR-VL-1.6 is PaddlePaddle's compact, roughly 0.9-billion-parameter vision-language model for document parsing, built on the ERNIE 4.5 line and specialized for OCR, table, formula, chart, and seal/stamp recognition plus text spotting rather than open-domain chat. It upgrades PaddleOCR-VL-1.5 with a region-aware data optimization framework that targets the earlier model's weak spots and a progressive post-training recipe combining curated data selection with reinforcement learning, while staying architecture-compatible with 1.5 for drop-in migration. The card reports a new state-of-the-art 96.33% on OmniDocBench v1.6. At under a billion parameters it runs on a single modest consumer GPU. Context length is 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in May 2026.

C4ai Command R V01

Cohere · 35.0B · runs from 15.9 GB

C4AI Command R is a 35-billion-parameter instruction-tuned model released by Cohere For AI in March 2024. It was built for conversational use with a particular focus on retrieval-augmented generation and tool use, including grounded answers that cite the source documents they draw on, and it was trained to work across ten major languages. It supports a 128K-token context window and is released under a Creative Commons Attribution-NonCommercial 4.0 license, so commercial use is not permitted. At roughly 35 billion parameters, 4-bit quantization needs around 20GB of memory, within reach of a single 24GB consumer GPU.

WhiteRabbitNeo 13B V1

WhiteRabbitNeo · 13B · runs from 7.5 GB

WhiteRabbitNeo 13B V1 is a 13B-parameter open language model from WhiteRabbitNeo. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Laguna XS.2

poolside · 33.4B · runs from 14.6 GB

Laguna XS.2 is a 33.4B-parameter open language model from poolside. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

CodeLlama 7B HF

Meta · 6.7B · runs from 4.2 GB

CodeLlama 7B HF is a 6.7B-parameter open language model from Meta in the Code Llama family. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

SmolLM 135M

Hugging Face · 135M · runs from 0.4 GB

SmolLM 135M is the original first-generation small language model from Hugging Face, designed to push the boundaries of what is achievable at extremely low parameter counts. With just 135 million parameters, it was a pioneering effort in making capable language models accessible on the most resource-constrained hardware. While the SmolLM2 and SmolLM3 families have since surpassed it in quality, the original SmolLM 135M remains a useful reference point for research and a practical option for ultra-lightweight deployment scenarios where every megabyte of memory counts.