All LLM Models

Browse 1475 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

OTel 2.0 LLM 31B IT

farbodtavakkoli · 31.3B · runs from 14.9 GB

OTel 2.0 LLM 31B IT is a 31.3B-parameter open language model from farbodtavakkoli. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.5 35B A3B

Alibaba · 36.0B · runs from 15.7 GB

Qwen3.5 35B A3B is Alibaba's mixture-of-experts vision-language model in the Qwen3.5 line, totaling 36 billion parameters with about 3 billion active per token (the A3B in its name). It accepts text, images, and video as input, using a hybrid linear-attention and sparse-MoE architecture built for efficiency. Only the active parameters compute per token, so generation is faster than a same-sized dense model, though all 36 billion weights must fit in memory, calling for a high-end 24-32GB-class GPU once quantized. The model offers a 262K token context window for long documents or multi-turn chats. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in February 2026 as part of Alibaba's push toward efficient multimodal MoE models.

Llava 1.5 7B HF

llava-hf · 7.1B · runs from 3.3 GB

LLaVA-1.5-7B is a 7-billion-parameter vision-language chat model made by fine-tuning Vicuna-7B, itself based on Llama 2, on GPT-generated multimodal instruction data, paired with a CLIP ViT-L/14 vision encoder via an MLP projector. It answers questions about images, describes visual content, and follows multi-turn multimodal instructions, and was one of the first widely used open vision-language chat models. At 7B it runs comfortably on a single consumer GPU once quantized. Context length is limited to 4,096 tokens, short by current standards given its 2023-era Llama 2 backbone. It is released under the Llama 2 Community License, permitting commercial use with some restrictions, including a separate license for very large companies; published in December 2023. Unlike newer small vision-language models, LLaVA-1.5 uses a simple architecture with no token-compression for image patches.

OTel LLM E4B IT

farbodtavakkoli · 4B · runs from 8.5 GB

OTel LLM E4B IT is a 4B-parameter open language model from farbodtavakkoli. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

H2ovl Mississippi 2B

h2oai · 2.2B · runs from 4.7 GB

H2OVL-Mississippi-2B is H2O.ai's 2.15-billion-parameter vision-language model, extending their H2O-Danube2-1.8B chat model with a vision encoder for multimodal tasks. It handles image captioning, visual question answering, and document understanding, positioned as an efficient, general-purpose OCR and document-AI model. H2O.ai trained it on 17 million image-text pairs and reports it competing with other 2B-class models like Qwen2-VL-2B and InternVL2-2B on public benchmarks. At this size, it runs easily on a single consumer GPU, even unquantized. Context length is 8,192 tokens, inherited from its Danube2 language backbone. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2024, it is part of H2O.ai's Mississippi family of open document-AI and OCR-focused vision-language models.

DeepSeek V4 Flash DSpark

DeepSeek · 165.3B · runs from 70.6 GB

DeepSeek V4 Flash DSpark packs roughly 165 billion parameters into a Flash-designated variant of DeepSeek's newer V4 line, aimed at faster inference than the standard release. It is a chat-oriented model intended for general dialogue and instruction-following tasks rather than a specialized one, continuing DeepSeek's open-weight release pattern. It supports a 1M token context window, well suited to long-document and long-context workloads that need to retain a great deal of information in a single pass. It is released under the MIT license for broad local and commercial use. At this parameter count, DeepSeek V4 Flash DSpark needs multi-GPU or server-class hardware to run locally, even once quantized.

OTel LLM 27B IT

farbodtavakkoli · 27B · runs from 13.1 GB

OTel LLM 27B IT is a 27B-parameter open language model from farbodtavakkoli. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 2 7B HF

Meta · 6.7B · runs from 3.1 GB

Meta Llama 2 7B is a 6.7-billion parameter base (pretrained) language model from Meta's Llama 2 generation, provided in Hugging Face Transformers format. It was trained on 2 trillion tokens with a 4K token context window and represented a significant step in openly available large language models when released. As a base model, it is designed for further fine-tuning and research rather than direct chat use. While superseded by Llama 3 and later releases in terms of benchmark performance, Llama 2 7B remains widely used in the research community and as a baseline for comparison. Released under the Llama 2 Community License.

Phi 3.5 Vision Instruct

Microsoft · 4.1B · runs from 2.9 GB

Phi 3.5 Vision Instruct is Microsoft's 4.1-billion-parameter multimodal model, pairing an image encoder and connector with the Phi-3 Mini language model to handle text and code alongside pictures. It suits visual question answering, chart and table reading, document OCR, and comparing details across multiple images in one prompt. Its small size suits laptops and modest consumer GPUs, running smoothly even on limited hardware once quantized. The model supports a 128K token context window, enough for lengthy documents. It is released under the MIT license, one of the most permissive options available, allowing unrestricted commercial and research use. Published in August 2024, it was trained on roughly 500 billion tokens of synthetic and filtered web data.

InternVL2 2B

OpenGVLab · 2.2B · runs from 4.8 GB

InternVL2-2B is OpenGVLab's 2.2-billion-parameter instruction-tuned vision-language model, part of the InternVL 2.0 family that ranges from 1 billion to 108 billion parameters. It pairs the InternViT-300M-448px vision encoder with the internlm2-chat-1.8b language model through an MLP projector, and is trained to handle document and chart comprehension, infographics QA, scene-text and OCR tasks, scientific and mathematical problem solving, and multi-image or video input. Compared with earlier Mini-InternVL models it adds support for long texts, multiple images, and video within the same training context. At just over 2 billion parameters, it runs comfortably on a single consumer GPU or even weaker hardware. Context length is 8,192 tokens, matching its 8k training context window. It is released under the MIT license, permitting unrestricted commercial and research use. It was published in June 2024, and has since been superseded by the InternVL2.5 and InternVL3 series.

Z Image Turbo

Tongyi-MAI · 6.2B · runs from 13.5 GB

Z Image Turbo is a 6.2B-parameter open language model from Tongyi-MAI. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen2.5 Coder 7B

Alibaba · 7.6B · runs from 3.6 GB

Qwen2.5 Coder 7B is a 7.6-billion parameter code-specialized base (pretrained) model from Alibaba Cloud's Qwen 2.5 Coder series. It is trained on a large dataset of source code and natural language but is not instruction-tuned, making it suitable for fine-tuning, code-related research, and custom downstream applications. The model supports a 128K token context window and runs efficiently on consumer GPUs. It serves as the foundation for the Qwen2.5 Coder 7B Instruct variant and community fine-tunes targeting specific programming languages or workflows. Released under the Apache 2.0 license.

GOT OCR2 0

StepFun · 716M · runs from 0.8 GB

GOT-OCR2.0 is a compact 716-million-parameter vision-language model built specifically for OCR, unifying plain-text OCR, formatted-text extraction (tables, formulas, and markup), and fine-grained region- or color-guided recognition in a single end-to-end model rather than a general chat assistant. It reads an image plus a task prompt and outputs recognized text, with an optional rendering mode that reconstructs the original layout as HTML. Unlike two-stage OCR pipelines that pair a separate detector with a text recognizer, GOT-OCR2.0 performs detection-free recognition end to end, and at under a billion parameters it runs comfortably on a single modest consumer GPU or even a CPU. Context length is 32,768 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in September 2024 by StepFun alongside its "General OCR Theory" paper.

Qwen1.5 MoE A2.7B

Alibaba · 14.3B · runs from 6.8 GB

Qwen1.5 MoE A2.7B is a Mixture of Experts (MoE) model from Alibaba Cloud's Qwen 1.5 generation, with 14.3 billion total parameters but only 2.7 billion active parameters per forward pass. The MoE architecture allows it to deliver performance closer to dense 7B models while requiring less compute during inference, as only a subset of expert layers are activated for each token. The model supports a 32K token context window and requires VRAM proportional to its total parameter count for loading, despite lower compute cost per token. It is an interesting architectural variant for users exploring efficient inference and MoE models locally. Released under a custom Qwen license.

Japanese GPT Neox Small

rinna · 204M · runs from 0.1 GB

Japanese GPT Neox Small is a 204M-parameter open language model from rinna. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Macbert4csc Base Chinese

shibing624 · 102M · runs from 0.2 GB

Macbert4csc Base Chinese is a 102M-parameter open language model from shibing624. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

GPT Neo 125M

EleutherAI · 150M · runs from 0.3 GB

GPT Neo 125M is a 150M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llava V1.6 Mistral 7B HF

llava-hf · 7.6B · runs from 16.6 GB

LLaVA-1.6 (also called LLaVA-NeXT) Mistral 7B is a 7.6-billion-parameter vision-language model that pairs a pretrained vision encoder with a Mistral-7B language backbone for multimodal chat, image captioning, and visual question answering. It improves on LLaVA-1.5 with higher and dynamic input image resolution, a larger and more diverse visual instruction-tuning mixture, and a commercially friendlier base model, together boosting its OCR and common-sense reasoning. This checkpoint is the Hugging Face Transformers-format port of the original research release, hosted under the community llava-hf organization rather than the original authors' account. At 7.6 billion parameters it fits comfortably on a single consumer GPU, especially once quantized. Context length is 32,768 tokens, inherited from the underlying Mistral-7B-Instruct-v0.2 backbone. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in February 2024, alongside sibling LLaVA-NeXT checkpoints built on Vicuna and Nous-Hermes-2-Yi-34B backbones.

Dots.mocr

dots-studio · 3.0B · runs from 1.6 GB

Dots.mocr is a 3-billion-parameter vision-language model from rednote-hilab, published on Hugging Face under the dots-studio organization, built specifically for OCR and document understanding rather than general chat. It unifies layout detection, text recognition, table and formula extraction, and reading-order recovery in one model, and can also convert charts, UI screenshots, and scientific figures directly into SVG code (a companion dots.mocr-svg checkpoint is tuned specifically for that task). On document-parsing benchmarks such as OmniDocBench and olmOCR-Bench it scores ahead of same-size competitors like MonkeyOCR-pro-3B and PaddleOCR-VL, and vLLM has integrated it natively since version 0.11.0. At just 3 billion parameters, it runs comfortably on a single consumer GPU. Context length is 131,072 tokens. It is released under the MIT license, with no usage restrictions. It was published in March 2026.

Chatglm2 6B

Z.ai · 6B · runs from 2.8 GB

ChatGLM2 6B, released in mid-2023, is a 6-billion-parameter chat model from Z.ai and an early entry in the GLM family, predating the newer GLM 5 line. It was built for general conversational use and was among the earlier openly available bilingual chat models of its generation, aimed at both English and Chinese dialogue. With a 32K token context window, it can handle moderately long conversations and documents, respectable for a model of its era. At 6 billion parameters, ChatGLM2 6B runs comfortably on modest consumer GPUs, making it accessible for local inference without specialized or high-end hardware.

Gemma4 E4b Claims Comparison

k-chirkunov · 7.9B · runs from 17.5 GB

Gemma4 E4b Claims Comparison is a 7.9B-parameter open language model from k-chirkunov in the Gemma 4 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Granite 4.1 3B

IBM · 3.4B · runs from 1.9 GB

Granite 4.1 3B is a 3.4B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma4 12B QAT Uncensored HauhauCS Balanced

HauhauCS · 12B · runs from 5.6 GB

Gemma4 12B QAT Uncensored HauhauCS Balanced is a 12B-parameter open language model from HauhauCS in the Gemma 4 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

VisionRoleplayChat

GPT OSS 20B NPU2

FastFlowLM · 20B · runs from 8.9 GB

GPT OSS 20B NPU2 is a 20B-parameter open language model from FastFlowLM in the GPT-OSS family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Vlt5 Base Keywords

Voicelab · 275M · runs from 0.6 GB

Vlt5 Base Keywords is a 275M-parameter open language model from Voicelab. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.8 27B DFlash2

incoai · 27B · runs from 11.8 GB

Qwen3.8 27B DFlash2 is a 27B-parameter open language model from incoai in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

GPT J 6B

EleutherAI · 6B · runs from 2.8 GB

GPT J 6B is a 6B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

GigaChat3.1 Audio 10B A1.8B

ai-sage · 10B · runs from 20.6 GB

GigaChat3.1 Audio 10B A1.8B is a 10B-parameter open language model from ai-sage. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Xflux Text Encoders

XLabs-AI · 4.8B · runs from 10.5 GB

Xflux Text Encoders is a 4.8B-parameter open language model from XLabs-AI. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3 4B Base Dapo V4

ReliquaryForge · 4.0B · runs from 2.2 GB

Qwen3 4B Base Dapo V4 is a 4.0B-parameter open language model from ReliquaryForge in the Qwen 3 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.