All LLM Models

Browse 982 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Phi 3 Mini 128k Instruct

Microsoft · 3.8B · runs from 2.7 GB

Phi 3 Mini 128k Instruct is a 3.8B-parameter open language model from Microsoft in the Phi 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 4 12B IT Assistant

Google · 12B · runs from 5.4 GB

Gemma 4 12B IT Assistant is a 12B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Phi Tiny MoE Instruct

Microsoft · 3.8B · runs from 2.2 GB

Phi Tiny MoE Instruct is a 3.8B-parameter open language model from Microsoft in the Phi family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Nemotron Labs Diffusion 3B

NVIDIA · 3.8B · runs from 2.1 GB

Nemotron Labs Diffusion 3B is a 3.8B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

A.X 4.0 Light

skt · 7.3B · runs from 2.4 GB

A.X 4.0 Light is a 7.3B-parameter open language model from skt. It supports a context window of up to 16,384 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Tiny Mixtral

TitanML · 247M · runs from 0.4 GB

Tiny Mixtral is a 247M-parameter open language model from TitanML in the Mixtral family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 3.1 Nemotron Nano 8B V1

NVIDIA · 8B · runs from 2.8 GB

Llama 3.1 Nemotron Nano 8B is an 8-billion parameter chat model by NVIDIA, a compact entry in the Nemotron family derived from Meta's Llama 3.1 architecture. It applies NVIDIA's alignment and fine-tuning techniques to deliver improved response quality over the base Llama 3.1 8B Instruct model at the same parameter count. The model runs on consumer GPUs with 8GB or more of VRAM and supports a 128K token context window. Its small footprint and NVIDIA-tuned quality make it a practical option for local inference on mainstream hardware.

Baichuan 13B Base

baichuan-inc · 13B · runs from 6.1 GB

Baichuan-13B-Base is Baichuan Intelligence's 13-billion-parameter pretrained base model, the follow-up to Baichuan-7B, trained on 1.4 trillion tokens of bilingual Chinese/English data, about 40% more than LLaMA-13B at the time. It replaces the usual rotary position embeddings with ALiBi linear position biasing, which the authors report gives roughly 31.6% faster token generation than a comparable LLaMA-13B. Official int8 and int4 quantized versions are provided so the model deploys on consumer GPUs such as the RTX 3090 with little accuracy loss. A separately released Baichuan-13B-Chat provides the aligned, conversational counterpart to this base checkpoint. Context length is 4,096 tokens. It is released under a custom Community License for the Baichuan-13B Model: free for academic research, with commercial use requiring written authorization from Baichuan via email request. It was published in July 2023, an early bilingual open model that predates the Baichuan2 series.

Baichuan2 13B Base

baichuan-inc · 13B · runs from 6.1 GB

Baichuan2-13B-Base is Baichuan Intelligence's 13-billion-parameter base language model — pretrained, not instruction-tuned — from the second-generation Baichuan series, trained on 2.6 trillion tokens of high-quality Chinese and English text. At release it topped same-size open models on Chinese and English benchmarks spanning general knowledge, law, medicine, math, code, and multilingual translation; a separate Baichuan2-13B-Chat model built on top of it adds instruction and safety tuning. As a 13-billion-parameter dense model, it fits on a single consumer GPU once quantized. It is released under a custom Baichuan2 community license: free for academic research and for commercial use once a developer applies for and receives a free commercial license from Baichuan by email. It was published in September 2023.

Qwen3.5 4B

Alibaba · 4.7B · runs from 2.5 GB

Qwen3.5-4B is Alibaba's dense 4-billion-parameter vision-language model from the Qwen3.5 generation, sharing the same hybrid Gated DeltaNet and gated-attention architecture as its larger siblings. This is the post-trained, instruction-tuned release, able to process images together with text and handle general chat, coding assistance, and agentic tool use. It has a 262,144-token native context window, extensible up to roughly 1,010,000 tokens, and is released under the Apache 2.0 license. At around 4.7 billion parameters, 4-bit quantization needs under 3GB of memory, so it runs easily on almost any modern GPU or laptop.

Opt 125M

Meta · 125M · runs from 0.3 GB

Meta OPT 125M is a 125-million parameter language model from Meta's Open Pre-trained Transformer (OPT) project. Released in 2022, it was part of Meta's effort to provide the research community with openly available large language models that replicate the performance of GPT-3 class models at various scales. As one of the smallest models in the OPT family, the 125M variant is primarily useful for research, experimentation, and educational purposes. It can run on virtually any hardware, including CPU-only setups. While significantly less capable than modern models, it remains a useful reference point in LLM research.

Llava 1.5 7B HF

llava-hf · 7.1B · runs from 3.3 GB

LLaVA-1.5-7B is a 7-billion-parameter vision-language chat model made by fine-tuning Vicuna-7B, itself based on Llama 2, on GPT-generated multimodal instruction data, paired with a CLIP ViT-L/14 vision encoder via an MLP projector. It answers questions about images, describes visual content, and follows multi-turn multimodal instructions, and was one of the first widely used open vision-language chat models. At 7B it runs comfortably on a single consumer GPU once quantized. Context length is limited to 4,096 tokens, short by current standards given its 2023-era Llama 2 backbone. It is released under the Llama 2 Community License, permitting commercial use with some restrictions, including a separate license for very large companies; published in December 2023. Unlike newer small vision-language models, LLaVA-1.5 uses a simple architecture with no token-compression for image patches.

OTel LLM E4B IT

farbodtavakkoli · 4B · runs from 8.5 GB

OTel LLM E4B IT is a 4B-parameter open language model from farbodtavakkoli. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

H2ovl Mississippi 2B

h2oai · 2.2B · runs from 4.7 GB

H2OVL-Mississippi-2B is H2O.ai's 2.15-billion-parameter vision-language model, extending their H2O-Danube2-1.8B chat model with a vision encoder for multimodal tasks. It handles image captioning, visual question answering, and document understanding, positioned as an efficient, general-purpose OCR and document-AI model. H2O.ai trained it on 17 million image-text pairs and reports it competing with other 2B-class models like Qwen2-VL-2B and InternVL2-2B on public benchmarks. At this size, it runs easily on a single consumer GPU, even unquantized. Context length is 8,192 tokens, inherited from its Danube2 language backbone. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. Published in October 2024, it is part of H2O.ai's Mississippi family of open document-AI and OCR-focused vision-language models.

Llama 2 7B HF

Meta · 6.7B · runs from 3.1 GB

Meta Llama 2 7B is a 6.7-billion parameter base (pretrained) language model from Meta's Llama 2 generation, provided in Hugging Face Transformers format. It was trained on 2 trillion tokens with a 4K token context window and represented a significant step in openly available large language models when released. As a base model, it is designed for further fine-tuning and research rather than direct chat use. While superseded by Llama 3 and later releases in terms of benchmark performance, Llama 2 7B remains widely used in the research community and as a baseline for comparison. Released under the Llama 2 Community License.

Phi 3.5 Vision Instruct

Microsoft · 4.1B · runs from 2.9 GB

Phi 3.5 Vision Instruct is Microsoft's 4.1-billion-parameter multimodal model, pairing an image encoder and connector with the Phi-3 Mini language model to handle text and code alongside pictures. It suits visual question answering, chart and table reading, document OCR, and comparing details across multiple images in one prompt. Its small size suits laptops and modest consumer GPUs, running smoothly even on limited hardware once quantized. The model supports a 128K token context window, enough for lengthy documents. It is released under the MIT license, one of the most permissive options available, allowing unrestricted commercial and research use. Published in August 2024, it was trained on roughly 500 billion tokens of synthetic and filtered web data.

InternVL2 2B

OpenGVLab · 2.2B · runs from 4.8 GB

InternVL2-2B is OpenGVLab's 2.2-billion-parameter instruction-tuned vision-language model, part of the InternVL 2.0 family that ranges from 1 billion to 108 billion parameters. It pairs the InternViT-300M-448px vision encoder with the internlm2-chat-1.8b language model through an MLP projector, and is trained to handle document and chart comprehension, infographics QA, scene-text and OCR tasks, scientific and mathematical problem solving, and multi-image or video input. Compared with earlier Mini-InternVL models it adds support for long texts, multiple images, and video within the same training context. At just over 2 billion parameters, it runs comfortably on a single consumer GPU or even weaker hardware. Context length is 8,192 tokens, matching its 8k training context window. It is released under the MIT license, permitting unrestricted commercial and research use. It was published in June 2024, and has since been superseded by the InternVL2.5 and InternVL3 series.

Qwen2.5 Coder 7B

Alibaba · 7.6B · runs from 3.6 GB

Qwen2.5 Coder 7B is a 7.6-billion parameter code-specialized base (pretrained) model from Alibaba Cloud's Qwen 2.5 Coder series. It is trained on a large dataset of source code and natural language but is not instruction-tuned, making it suitable for fine-tuning, code-related research, and custom downstream applications. The model supports a 128K token context window and runs efficiently on consumer GPUs. It serves as the foundation for the Qwen2.5 Coder 7B Instruct variant and community fine-tunes targeting specific programming languages or workflows. Released under the Apache 2.0 license.

GOT OCR2 0

StepFun · 716M · runs from 0.8 GB

GOT-OCR2.0 is a compact 716-million-parameter vision-language model built specifically for OCR, unifying plain-text OCR, formatted-text extraction (tables, formulas, and markup), and fine-grained region- or color-guided recognition in a single end-to-end model rather than a general chat assistant. It reads an image plus a task prompt and outputs recognized text, with an optional rendering mode that reconstructs the original layout as HTML. Unlike two-stage OCR pipelines that pair a separate detector with a text recognizer, GOT-OCR2.0 performs detection-free recognition end to end, and at under a billion parameters it runs comfortably on a single modest consumer GPU or even a CPU. Context length is 32,768 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in September 2024 by StepFun alongside its "General OCR Theory" paper.

Qwen1.5 MoE A2.7B

Alibaba · 14.3B · runs from 6.8 GB

Qwen1.5 MoE A2.7B is a Mixture of Experts (MoE) model from Alibaba Cloud's Qwen 1.5 generation, with 14.3 billion total parameters but only 2.7 billion active parameters per forward pass. The MoE architecture allows it to deliver performance closer to dense 7B models while requiring less compute during inference, as only a subset of expert layers are activated for each token. The model supports a 32K token context window and requires VRAM proportional to its total parameter count for loading, despite lower compute cost per token. It is an interesting architectural variant for users exploring efficient inference and MoE models locally. Released under a custom Qwen license.

Japanese GPT Neox Small

rinna · 204M · runs from 0.1 GB

Japanese GPT Neox Small is a 204M-parameter open language model from rinna. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Macbert4csc Base Chinese

shibing624 · 102M · runs from 0.2 GB

Macbert4csc Base Chinese is a 102M-parameter open language model from shibing624. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

GPT Neo 125M

EleutherAI · 150M · runs from 0.3 GB

GPT Neo 125M is a 150M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Dots.mocr

dots-studio · 3.0B · runs from 1.6 GB

Dots.mocr is a 3-billion-parameter vision-language model from rednote-hilab, published on Hugging Face under the dots-studio organization, built specifically for OCR and document understanding rather than general chat. It unifies layout detection, text recognition, table and formula extraction, and reading-order recovery in one model, and can also convert charts, UI screenshots, and scientific figures directly into SVG code (a companion dots.mocr-svg checkpoint is tuned specifically for that task). On document-parsing benchmarks such as OmniDocBench and olmOCR-Bench it scores ahead of same-size competitors like MonkeyOCR-pro-3B and PaddleOCR-VL, and vLLM has integrated it natively since version 0.11.0. At just 3 billion parameters, it runs comfortably on a single consumer GPU. Context length is 131,072 tokens. It is released under the MIT license, with no usage restrictions. It was published in March 2026.

Chatglm2 6B

Z.ai · 6B · runs from 2.8 GB

ChatGLM2 6B, released in mid-2023, is a 6-billion-parameter chat model from Z.ai and an early entry in the GLM family, predating the newer GLM 5 line. It was built for general conversational use and was among the earlier openly available bilingual chat models of its generation, aimed at both English and Chinese dialogue. With a 32K token context window, it can handle moderately long conversations and documents, respectable for a model of its era. At 6 billion parameters, ChatGLM2 6B runs comfortably on modest consumer GPUs, making it accessible for local inference without specialized or high-end hardware.

Granite 4.1 3B

IBM · 3.4B · runs from 1.9 GB

Granite 4.1 3B is a 3.4B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma4 12B QAT Uncensored HauhauCS Balanced

HauhauCS · 12B · runs from 5.6 GB

Gemma4 12B QAT Uncensored HauhauCS Balanced is a 12B-parameter open language model from HauhauCS in the Gemma 4 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

VisionRoleplayChat

GPT OSS 20B NPU2

FastFlowLM · 20B · runs from 8.9 GB

GPT OSS 20B NPU2 is a 20B-parameter open language model from FastFlowLM in the GPT-OSS family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Vlt5 Base Keywords

Voicelab · 275M · runs from 0.6 GB

Vlt5 Base Keywords is a 275M-parameter open language model from Voicelab. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen3.8 27B DFlash2

incoai · 27B · runs from 11.8 GB

Qwen3.8 27B DFlash2 is a 27B-parameter open language model from incoai in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.