All LLM Models
Browse 1214 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
HunyuanOCR
Tencent · 1.1B · runs from 0.7 GB
713.3K
823HunyuanOCR is Tencent's 1.1-billion-parameter vision-language model built for OCR and document understanding, unifying document parsing, text spotting, information extraction, and text-image translation in one model rather than a general chat assistant. The current checkpoint, HunyuanOCR-1.5, adds DFlash speculative decoding, where a lightweight draft model proposes tokens that the main model verifies in one pass, cutting latency on long structured outputs like tables and formulas. It also ships GGUF weights for llama.cpp, running on CPUs and laptop or consumer GPUs, not just server-grade vLLM. At just over 1 billion parameters, it is light enough for a single modest consumer GPU. Context length is 131,072 tokens; pretraining supports image resolutions up to 4K. It is released under the Tencent Hunyuan Community License, a custom license permitting commercial use outside the EU, UK, and South Korea and below 100 million monthly active users. It was published in November 2025.
Qwen3.8 27B OBLITERATED Mythos Class Agentic
medismera · 27.8B · runs from 12.6 GB
Qwen3.8 27B OBLITERATED Mythos Class Agentic is a 27.8B-parameter open language model from medismera in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2 VL 7B Instruct
Alibaba · 8.3B · runs from 3.2 GB
Qwen2 VL 7B Instruct is Alibaba's 8.3-billion-parameter vision-language model from the original Qwen 2 generation, built to process images and video alongside text in a single conversation. It can describe images, answer visual questions, and reason over multi-image or video input, suiting it to multimodal chat and visual document tasks. At this parameter count, local inference is practical on a single mainstream or high-end consumer GPU once the weights are quantized, rather than requiring multi-GPU hardware. The model supports a 32K token context window, enough for moderate-length documents or multi-turn visual conversations. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in August 2024, it was later superseded by Qwen2.5-VL-7B-Instruct, which its own model card lists as its successor.
SmolVLM 256M Instruct
Hugging Face · 256M · runs from 0.4 GB
SmolVLM-256M Instruct is Hugging Face's 256-million-parameter vision-language model, built on a compact SmolLM2-135M language backbone paired with a small 93-million-parameter SigLIP vision encoder. Hugging Face calls it the smallest publicly available multimodal model, designed for image captioning, visual question answering, and basic text transcription from images rather than open-ended chat. Its size makes it practical to run even on CPUs or entry-level GPUs, and on-device deployment is a stated use case. Context length is limited to 8,192 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in January 2025. Compared to the larger 2.2B SmolVLM2 sibling, it uses a smaller vision encoder and more aggressive image-token compression to further cut memory and latency, at some cost to accuracy.
Granite 4.0 Micro
IBM · 3.4B · runs from 1.4 GB
Granite-4.0-Micro is IBM's 3-billion-parameter long-context instruct model, fine-tuned from Granite-4.0-Micro-Base on a mix of permissively licensed open datasets and IBM's own synthetic data, using supervised fine-tuning, reinforcement-learning alignment, and model merging. Unlike some of its Granite 4.0 siblings (H Micro, H Tiny MoE, H Small MoE), which mix in Mamba2 layers or mixture-of-experts routing, Granite-4.0-Micro is a plain decoder-only dense transformer with grouped-query attention, RoPE, SwiGLU, and RMSNorm. It targets enterprise use cases such as summarization, RAG, tool-calling, and code completion, and at 3B parameters it runs comfortably on a single consumer GPU, even quantized on a laptop. Context length is 131,072 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in October 2025.
LFM2.5 1.2B Thinking
Liquid AI · 1.2B · runs from 0.9 GB
LFM2.5 1.2B Thinking is a 1.2B-parameter open language model from Liquid AI in the LFM2.5 family. It supports a context window of up to 128,000 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
K2 Horizon 7B
IFM · 9.0B · runs from 4.4 GB
K2 Horizon 7B is a 9.0B-parameter open language model from IFM. It supports a context window of up to 524,288 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mistral 7B v0.1
Mistral AI · 7.2B · runs from 3.6 GB
Mistral 7B v0.1 is the original base model from Mistral AI that helped reshape expectations for small open-weight language models when it launched in late 2023. As a pretrained foundation model without instruction tuning, it is designed for fine-tuning, research, and custom downstream tasks rather than direct conversational use. With 7 billion parameters and support for grouped-query attention and sliding-window attention, it remains a popular starting point for practitioners building specialized models. Its modest VRAM requirements of roughly 6 GB at 4-bit quantization keep it accessible on a wide range of consumer GPUs.
DeepSeek R1 Distill Qwen 14B Abliterated v2
huihui-ai · 14.8B · runs from 7.0 GB
DeepSeek R1 Distill Qwen 14B Abliterated v2 is a 14.8B-parameter open language model from huihui-ai in the DeepSeek R1 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Granite 3.3 8B Instruct
IBM · 8.2B · runs from 2.9 GB
Granite 3.3 8B Instruct is a 8.2B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Chandra Ocr 2
datalab-to · 5.3B · runs from 2.7 GB
Chandra OCR 2 is Datalab's 5.3-billion-parameter vision-language model for OCR and document conversion, turning scanned pages and PDFs into markdown, HTML, or JSON while preserving layout. It runs on a Qwen3.5-based backbone mixing linear attention with periodic full-attention layers, and handles handwriting, forms, tables, and math across 90-plus languages. It is the second generation of Datalab's Chandra model, improved over the first release. At just over 5 billion parameters, it runs on a single consumer GPU once quantized. Context extends to 262,144 tokens, generous for multi-page documents. It carries a modified OpenRAIL-M license: free for research, personal use, and startups under $2 million in funding or revenue, but it bars competing against Datalab's own API, more restrictive than a standard permissive license. Published in March 2026, following the original Chandra.
Gemma 4 12B IT AEON Abliterated K4 BF16
AEON-7 · 12.0B · runs from 6.1 GB
Gemma 4 12B IT AEON Abliterated K4 BF16 is a 12.0B-parameter open language model from AEON-7 in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Falcon 40B Instruct
TII UAE · 40B · runs from 12.1 GB
Falcon-40B-Instruct is TII's 40-billion-parameter causal decoder-only chat model, fine-tuned from the Falcon-40B base model on the Baize chat dataset so it is ready to use as an assistant out of the box. Falcon-40B was, at release, the top-ranked fully open model on the Open LLM Leaderboard, and the architecture is optimized for inference with FlashAttention and multi-query attention. It is trained and used primarily in English, with some French capability, and TII recommends further safety evaluation before production use. At 40B parameters it needs a multi-GPU workstation in half precision, while a 4-bit quantization brings it within reach of a single high-memory GPU. License is Apache 2.0, permitting unrestricted commercial and research use. It was published in May 2023, one day after the Falcon-40B base model it is fine-tuned from.
Qwen2.5 Coder 14B
Alibaba · 14.8B · runs from 5.1 GB
Qwen2.5 Coder 14B is a 14.8B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
OvisOCR2
ATH-MaaS · 853M · runs from 0.6 GB
OvisOCR2 is a compact 853-million-parameter end-to-end vision-language model for page-level document OCR, built by post-training Qwen3.5-0.8B with a data engine that mixes real and synthetic documents and a multi-stage supervised fine-tuning, reinforcement learning, and OPD training recipe. Given a document page image it outputs a single Markdown document that reproduces the natural reading order, including body text, formulas rendered as LaTeX, tables as HTML, and captioned image regions, replacing traditional multi-stage OCR pipelines with one model. It set a new state of the art on the OmniDocBench v1.6 leaderboard, the first end-to-end model to top a benchmark long dominated by pipeline-based approaches, and also leads PureDocBench. At under a billion parameters it runs comfortably on a single consumer GPU or even a CPU. Context length is 262,144 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in July 2026.
Gemma 4 31B IT Heretic
coder3101 · 31.3B · runs from 10.2 GB
Gemma 4 31B IT Heretic is a 31.3B-parameter open language model from coder3101 in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Hermes 3 Llama 3.2 3B
Nous Research · 3B · runs from 1.6 GB
Hermes 3 Llama 3.2 3B is a 3-billion parameter instruction-tuned model by Nous Research, fine-tuned from Meta's Llama 3.2 3B base. It applies the Hermes training methodology to a compact model, targeting strong instruction following and conversational quality at minimal hardware cost. Despite its small size, this model benefits from the Hermes fine-tuning approach that emphasizes system prompt adherence and structured output. It can run on GPUs with as little as 4GB of VRAM when quantized, making it suitable for lightweight local deployments and resource-constrained environments.
Ornith 1.5 9B OBLITERATED
OBLITERATUS · 9.7B · runs from 4.7 GB
Ornith 1.5 9B OBLITERATED is a 9.7B-parameter open language model from OBLITERATUS in the Ornith family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 3.1 Nemotron 70B Instruct HF
NVIDIA · 70.6B · runs from 20.4 GB
Llama 3.1 Nemotron 70B Instruct is a 70-billion parameter chat model by NVIDIA, created by applying reinforcement learning from human feedback (RLHF) to Meta's Llama 3.1 70B base model. NVIDIA's Nemotron training pipeline focuses on improving helpfulness, accuracy, and response quality beyond the standard Llama instruction tuning. The model requires substantial VRAM for local inference, typically needing multi-GPU setups or high-end professional GPUs. In quantized formats it becomes accessible on workstation-class hardware. It is available in Hugging Face Transformers format and is supported by popular inference engines.
Rocinante X 12B V1
TheDrummer · 12.2B · runs from 4.5 GB
Rocinante X 12B V1 is a 12.2B-parameter open language model from TheDrummer. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron Cascade 2 30B A3B
NVIDIA · 31.6B · runs from 9.1 GB
Nemotron Cascade 2 30B A3B is a 31.6B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Phi 3 Medium 128k Instruct
Microsoft · 14.0B · runs from 4.6 GB
Phi-3-Medium-128K-Instruct is Microsoft's 14-billion-parameter instruction-tuned model in the Phi-3 family, trained on a mix of synthetic data and filtered high-quality web content chosen for reasoning density, then post-trained with supervised fine-tuning and direct preference optimization for instruction-following and safety. It is the long-context variant of Phi-3-Medium, alongside a 4K-context sibling, and is aimed at memory- and latency-constrained deployments needing strong code, math, and logical reasoning rather than frontier-scale serving. At 14B parameters, it runs on a single high-end consumer GPU once quantized. Context length is 131,072 tokens. It is released under the MIT license, permitting unrestricted commercial and research use, and was published in May 2024.
OpenHermes 2.5 Mistral 7B
Teknium · 7B · runs from 3.5 GB
OpenHermes 2.5 is a community-driven fine-tune of Mistral 7B created by Teknium, trained on over 900,000 entries of high-quality synthetic data generated primarily by GPT-4. It quickly became one of the most popular open chat models of its era, consistently topping community benchmarks for 7B-class models. For local users, it offers strong instruction-following, creative writing, and coding assistance in a package that runs comfortably on a single consumer GPU with 8 GB of VRAM.
Kimi VL A3B Instruct
Moonshot AI · 16.4B · runs from 5.3 GB
Kimi VL A3B Instruct is Moonshot AI's 16.4-billion-parameter mixture-of-experts vision-language model, with about 3 billion parameters active per token (the A3B in its name). Because only the active experts run for each token, it runs faster than a dense model of similar total size, though the full set of weights must still fit in memory. It pairs a native-resolution vision encoder with a language core built on Moonshot's Moonlight-16B-A3B model, supporting image and video understanding, OCR, multi-image reasoning, and agent-style tool use. At this size, local inference is practical on a single high-end consumer GPU once quantized. The model supports a 131K token context window, useful for long documents or video transcripts. It is released under the MIT license, one of the most permissive open licenses, and was published in April 2025.
Qwen3.6 35B A3B Abliterated V4
Bahushruth · 34.7B · runs from 12.1 GB
Qwen3.6 35B A3B Abliterated V4 is a 34.7B-parameter open language model from Bahushruth in the Qwen 3.6 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Internlm3 8B Instruct
InternLM · 8.8B · runs from 3.4 GB
Internlm3 8B Instruct is a 8.8B-parameter open language model from InternLM in the InternLM family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Huihui GPT OSS 20B BF16 Abliterated
huihui-ai · 20.9B · runs from 9.3 GB
Huihui GPT OSS 20B BF16 Abliterated is a 20.9B-parameter open language model from huihui-ai in the GPT-OSS family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Dolphin 2.9.1 Yi 1.5 34B
dphn · 34.4B · runs from 10.3 GB
Dolphin 2.9.1 Yi 1.5 34B is a 34.4-billion parameter chat model created by Eric Hartford's Dolphin project, fine-tuned from 01.AI's Yi 1.5 34B base. The Dolphin series is known for producing uncensored fine-tunes that remove alignment-based refusals, giving users more direct and unrestricted model responses. This model combines the strong bilingual capabilities of Yi 1.5 with Dolphin's open fine-tuning approach. It requires a GPU with at least 24GB of VRAM for quantized local inference and is popular among users who prefer models without built-in content restrictions.
DeepSeek OCR 2
DeepSeek · 3.4B · runs from 1.9 GB
DeepSeek OCR 2 is DeepSeek's compact vision-language model built for optical character recognition and document understanding, totaling 3.4 billion parameters with about 1.2 billion active per token through its mixture-of-experts design. Only the active parameters compute per token, keeping inference fast, though the full weight set still needs to fit in memory; at this size that's within reach of a consumer GPU or laptop once quantized. It uses a DeepEncoder V2 vision architecture that reasons over document layout semantically instead of scanning images in a fixed pattern. The model has an 8K token context window, suited to single-document OCR passes. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, and was published in January 2026 as DeepSeek's OCR successor.
ZDTaichu5.0 9B
TaichuAI · 9.8B · runs from 3.2 GB
ZDTaichu5.0-9B is TaichuAI's 9.8-billion-parameter multimodal foundation model, combining a Qwen3.5-9B language backbone with an NVIDIA C-RADIOv4-H vision encoder to accept text, images, and video at any resolution. Beyond general image, document, and OCR understanding, it is built for spatial reasoning (2D/3D relations, viewpoint and depth, affordances), embodied-AI planning, and multi-step agentic tool use, using an "Entropy-Gated Adaptive Recurrent Reasoning" mechanism that allocates extra recurrent computation to harder tokens. The card reports it leading spatial and agentic benchmarks among comparable 10B-scale open vision-language models. At under 10 billion parameters, it fits on a single consumer GPU once quantized. Context length is 131,072 tokens. It is released under the NVIDIA Open Model License Agreement, with the underlying Qwen3.5 component retaining its Apache 2.0 license, and was published in September 2026.