All LLM Models
Browse 1242 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.
Model List
Z Image Turbo
Tongyi-MAI · 6.2B · runs from 13.5 GB
682.3K
5.2KZ Image Turbo is a 6.2B-parameter open language model from Tongyi-MAI. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen2.5 Coder 7B
Alibaba · 7.6B · runs from 3.6 GB
Qwen2.5 Coder 7B is a 7.6-billion parameter code-specialized base (pretrained) model from Alibaba Cloud's Qwen 2.5 Coder series. It is trained on a large dataset of source code and natural language but is not instruction-tuned, making it suitable for fine-tuning, code-related research, and custom downstream applications. The model supports a 128K token context window and runs efficiently on consumer GPUs. It serves as the foundation for the Qwen2.5 Coder 7B Instruct variant and community fine-tunes targeting specific programming languages or workflows. Released under the Apache 2.0 license.
GOT OCR2 0
StepFun · 716M · runs from 0.8 GB
GOT-OCR2.0 is a compact 716-million-parameter vision-language model built specifically for OCR, unifying plain-text OCR, formatted-text extraction (tables, formulas, and markup), and fine-grained region- or color-guided recognition in a single end-to-end model rather than a general chat assistant. It reads an image plus a task prompt and outputs recognized text, with an optional rendering mode that reconstructs the original layout as HTML. Unlike two-stage OCR pipelines that pair a separate detector with a text recognizer, GOT-OCR2.0 performs detection-free recognition end to end, and at under a billion parameters it runs comfortably on a single modest consumer GPU or even a CPU. Context length is 32,768 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in September 2024 by StepFun alongside its "General OCR Theory" paper.
Qwen1.5 MoE A2.7B
Alibaba · 14.3B · runs from 6.8 GB
Qwen1.5 MoE A2.7B is a Mixture of Experts (MoE) model from Alibaba Cloud's Qwen 1.5 generation, with 14.3 billion total parameters but only 2.7 billion active parameters per forward pass. The MoE architecture allows it to deliver performance closer to dense 7B models while requiring less compute during inference, as only a subset of expert layers are activated for each token. The model supports a 32K token context window and requires VRAM proportional to its total parameter count for loading, despite lower compute cost per token. It is an interesting architectural variant for users exploring efficient inference and MoE models locally. Released under a custom Qwen license.
Japanese GPT Neox Small
rinna · 204M · runs from 0.1 GB
Japanese GPT Neox Small is a 204M-parameter open language model from rinna. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Macbert4csc Base Chinese
shibing624 · 102M · runs from 0.2 GB
Macbert4csc Base Chinese is a 102M-parameter open language model from shibing624. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GPT Neo 125M
EleutherAI · 150M · runs from 0.3 GB
GPT Neo 125M is a 150M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llava V1.6 Mistral 7B HF
llava-hf · 7.6B · runs from 16.6 GB
LLaVA-1.6 (also called LLaVA-NeXT) Mistral 7B is a 7.6-billion-parameter vision-language model that pairs a pretrained vision encoder with a Mistral-7B language backbone for multimodal chat, image captioning, and visual question answering. It improves on LLaVA-1.5 with higher and dynamic input image resolution, a larger and more diverse visual instruction-tuning mixture, and a commercially friendlier base model, together boosting its OCR and common-sense reasoning. This checkpoint is the Hugging Face Transformers-format port of the original research release, hosted under the community llava-hf organization rather than the original authors' account. At 7.6 billion parameters it fits comfortably on a single consumer GPU, especially once quantized. Context length is 32,768 tokens, inherited from the underlying Mistral-7B-Instruct-v0.2 backbone. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use. It was published in February 2024, alongside sibling LLaVA-NeXT checkpoints built on Vicuna and Nous-Hermes-2-Yi-34B backbones.
Dots.mocr
dots-studio · 3.0B · runs from 1.6 GB
Dots.mocr is a 3-billion-parameter vision-language model from rednote-hilab, published on Hugging Face under the dots-studio organization, built specifically for OCR and document understanding rather than general chat. It unifies layout detection, text recognition, table and formula extraction, and reading-order recovery in one model, and can also convert charts, UI screenshots, and scientific figures directly into SVG code (a companion dots.mocr-svg checkpoint is tuned specifically for that task). On document-parsing benchmarks such as OmniDocBench and olmOCR-Bench it scores ahead of same-size competitors like MonkeyOCR-pro-3B and PaddleOCR-VL, and vLLM has integrated it natively since version 0.11.0. At just 3 billion parameters, it runs comfortably on a single consumer GPU. Context length is 131,072 tokens. It is released under the MIT license, with no usage restrictions. It was published in March 2026.
Chatglm2 6B
Z.ai · 6B · runs from 2.8 GB
ChatGLM2 6B, released in mid-2023, is a 6-billion-parameter chat model from Z.ai and an early entry in the GLM family, predating the newer GLM 5 line. It was built for general conversational use and was among the earlier openly available bilingual chat models of its generation, aimed at both English and Chinese dialogue. With a 32K token context window, it can handle moderately long conversations and documents, respectable for a model of its era. At 6 billion parameters, ChatGLM2 6B runs comfortably on modest consumer GPUs, making it accessible for local inference without specialized or high-end hardware.
Gemma4 E4b Claims Comparison
k-chirkunov · 7.9B · runs from 17.5 GB
Gemma4 E4b Claims Comparison is a 7.9B-parameter open language model from k-chirkunov in the Gemma 4 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Granite 4.1 3B
IBM · 3.4B · runs from 1.9 GB
Granite 4.1 3B is a 3.4B-parameter open language model from IBM in the Granite family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Gemma4 12B QAT Uncensored HauhauCS Balanced
HauhauCS · 12B · runs from 5.6 GB
Gemma4 12B QAT Uncensored HauhauCS Balanced is a 12B-parameter open language model from HauhauCS in the Gemma 4 family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GPT OSS 20B NPU2
FastFlowLM · 20B · runs from 8.9 GB
GPT OSS 20B NPU2 is a 20B-parameter open language model from FastFlowLM in the GPT-OSS family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Vlt5 Base Keywords
Voicelab · 275M · runs from 0.6 GB
Vlt5 Base Keywords is a 275M-parameter open language model from Voicelab. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3.8 27B DFlash2
incoai · 27B · runs from 11.8 GB
Qwen3.8 27B DFlash2 is a 27B-parameter open language model from incoai in the Qwen 3.8 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GPT J 6B
EleutherAI · 6B · runs from 2.8 GB
GPT J 6B is a 6B-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
GigaChat3.1 Audio 10B A1.8B
ai-sage · 10B · runs from 20.6 GB
GigaChat3.1 Audio 10B A1.8B is a 10B-parameter open language model from ai-sage. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Xflux Text Encoders
XLabs-AI · 4.8B · runs from 10.5 GB
Xflux Text Encoders is a 4.8B-parameter open language model from XLabs-AI. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Qwen3 4B Base Dapo V4
ReliquaryForge · 4.0B · runs from 2.2 GB
Qwen3 4B Base Dapo V4 is a 4.0B-parameter open language model from ReliquaryForge in the Qwen 3 family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Apertus V1.5 8B
swiss-ai · 8.9B · runs from 4.2 GB
Apertus V1.5 8B is an 8.9-billion-parameter multimodal model from the Swiss AI Initiative, a collaboration between ETH Zurich, EPFL, and the Swiss National Supercomputing Centre. Extended from the text-only Apertus 1 through continued pretraining, it accepts images alongside text and can process spoken audio experimentally, letting it reason about visual content in a chat setting. Its compact size suits local deployment on a single consumer GPU once quantized. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use, alongside a public acceptable-use policy governing deployment. Published in July 2026, Apertus V1.5 stands out for being fully open in a way few large models are: not just the weights but the training data, code, and development process are published, with an optional reasoning mode for harder prompts.
Opt 350M
Meta · 350M · runs from 0.8 GB
Meta OPT 350M is a 350-million parameter language model from Meta's Open Pre-trained Transformer (OPT) project, released in 2022 as part of a suite of models ranging from 125M to 175B parameters. It was designed to provide researchers with open access to models comparable to GPT-3 at various scales. The 350M variant runs on minimal hardware and is suitable for research, prototyping, and educational use. While it has been surpassed by modern architectures in terms of capability, it remains a lightweight option for basic text generation experiments and as a benchmark baseline.
DeepSeek v2 Lite
DeepSeek · 15.7B · runs from 7.4 GB
DeepSeek V2 Lite is a compact mixture-of-experts model with 15.7 billion total parameters, designed to deliver a strong quality-to-compute ratio for general chat and instruction following. It uses the same innovative MLA (Multi-Head Latent Attention) architecture as the larger V2, which reduces memory requirements during inference. With its modest parameter count, V2 Lite runs comfortably on a single consumer GPU, making it accessible to users who want to try DeepSeek's MoE approach without needing specialized hardware. It handles everyday conversational tasks, summarization, and light analysis well, offering a practical entry point into the DeepSeek model family.
Openai GPT
OpenAI · 120M · runs from 0.1 GB
OpenAI GPT is the original 2018 transformer-based language model that started the GPT lineage, based on the paper "Improving Language Understanding by Generative Pre-Training." At just 120 million parameters, it is a historically significant model that demonstrated the power of unsupervised pretraining followed by supervised fine-tuning. This model is primarily of academic and historical interest today. It runs on essentially any hardware and can be useful for educational exploration of transformer architectures, but it should not be compared to modern instruction-tuned models in terms of practical capability.
The GuageLLM 23M
Hai929 · 23M · runs from 0.0 GB
The GuageLLM 23M is a 23M-parameter open language model from Hai929. It supports a context window of up to 64 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 7B
huggyllama · 6.7B · runs from 3.1 GB
This is a community reupload of Meta's original Llama 1 7B model, published by the huggyllama account on Hugging Face. The original Llama 1 was a 6.7-billion parameter base model released by Meta in early 2023, trained on 1 trillion tokens of publicly available data. It pioneered the wave of open-weight large language models. As a first-generation Llama model, it has been superseded by Llama 2 and Llama 3 in terms of quality and capability. It remains of historical and research interest as the model that catalyzed the open-source LLM ecosystem. This upload provides convenient access in Hugging Face Transformers format.
LLaDA2.0 Mini
Inclusion AI · 16.3B · runs from 7.3 GB
LLaDA2.0 Mini is a 16.3B-parameter open language model from Inclusion AI. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Hyenadna Medium 450k Seqlen HF
LongSafari · 28M · runs from 0.1 GB
Hyenadna Medium 450k Seqlen HF is a 28M-parameter open language model from LongSafari. It supports a context window of up to 450,002 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
TinyStories 1M
roneneldan · 1M · runs from 0.0 GB
TinyStories 1M is a 1M-parameter open language model from roneneldan. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Zamba2 1.2B Instruct
Zyphra · 1.2B · runs from 1.4 GB
Zamba2 1.2B Instruct is a 1.2B-parameter open language model from Zyphra. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.