All LLM Models

Browse 1214 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

PowerMoE 3B

ibm-research · 3.4B · runs from 1.7 GB

PowerMoE-3B is IBM Research's 3.4-billion-parameter sparse Mixture-of-Experts language model, built for general text generation, reasoning, and code rather than a specific vision or audio modality. It routes each token to 8 of 40 experts, activating roughly 800 million parameters per token according to IBM, keeping decoding fast even though the full set of experts still has to be held in memory. At this size it fits comfortably on a single mainstream consumer GPU once quantized. Context length is limited to 4,096 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in August 2024. Its defining feature is the Power learning-rate scheduler IBM used to train it, which the team reports lets an 800-million-active-parameter model match dense models with roughly twice as many active parameters.

Seed OSS 36B Base

ByteDance-Seed · 36.2B · runs from 15.9 GB

Seed-OSS-36B-Base is ByteDance's 36-billion-parameter open-source base language model, a dense transformer using GQA attention, RMSNorm, and SwiGLU, designed as a foundation for long-context, reasoning, and agentic downstream use. It is a pretrained model, not instruction-tuned, though this particular checkpoint's pretraining mix includes synthetic instruction data (ByteDance separately released a Base-woSyn variant trained without it, for researchers who want a foundation model unaffected by synthetic instruction data). Despite training on only 12 trillion tokens, it scores competitively against larger open pretrained models on knowledge, reasoning, and math benchmarks. At 36B parameters it needs a multi-GPU workstation or heavy quantization to run locally. Context length is 524,288 tokens, trained natively rather than extended after the fact. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in August 2025 alongside the instruction-tuned Seed-OSS-36B-Instruct.

LocoTrainer 4B

LocoreMind · 4.0B · runs from 2.2 GB

LocoTrainer 4B is a 4.0B-parameter open language model from LocoreMind. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatCodeFunctions

Prometheus 7B V2.0

prometheus-eval · 7.2B · runs from 3.6 GB

Prometheus 7B V2.0 is a specialized judge model trained by prometheus-eval to evaluate the quality of outputs from other language models. At 7.2 billion parameters, it is designed to score and critique LLM responses against custom rubrics, making it a valuable tool for automated evaluation pipelines and benchmarking. Unlike general-purpose chat models, Prometheus is purpose-built for assessment tasks. It can provide structured feedback on dimensions like helpfulness, accuracy, and coherence. Useful for researchers, developers building LLM applications, and anyone who needs consistent automated evaluation without relying on paid API calls to frontier models. Runs comfortably on most modern GPUs with 8 GB or more of VRAM.

ReaderLM v2

jinaai · 1.5B · runs from 1.0 GB

ReaderLM v2 is a 1.5B-parameter open language model from jinaai. It supports a context window of up to 512,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Pebble 10M Chat

basically-ai · 11M · runs from 0.0 GB

Pebble 10M Chat is a 11M-parameter open language model from basically-ai. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

CodeQwen1.5 7B

Alibaba · 7.3B · runs from 3.5 GB

CodeQwen1.5-7B is Alibaba's 7.3-billion-parameter code-specialized base language model, built on the Qwen1.5 architecture and pretrained on 3 trillion tokens of code data covering 92 programming languages. It uses group-query attention for efficient inference and is a raw pretrained model, not tuned for chat; Alibaba advises against using it directly for conversation and instead for fine-tuning, code infilling, and code generation. The card highlights strong text-to-SQL and bug-fixing performance for its size. At 7B parameters it fits comfortably on a single consumer GPU, even less once quantized. Context length is 65,536 tokens. It is released under a custom Tongyi Qianwen Research License, free for academic research, with commercial use requiring direct contact with Alibaba. It was published in April 2024.

Qwen1.5 7B Chat

Alibaba · 7.7B · runs from 4.7 GB

Qwen1.5 7B Chat is a 7.7B-parameter open language model from Alibaba in the Qwen family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 3 4B T1 IT

twinkle-ai · 3.9B · runs from 2.3 GB

Gemma 3 4B T1 IT is a 3.9B-parameter open language model from twinkle-ai in the Gemma 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

SmolLM2 360M

Hugging Face · 362M · runs from 0.5 GB

SmolLM2 360M is a 362M-parameter open language model from Hugging Face in the SmolLM family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Amber

LLM360 · 6.7B · runs from 3.2 GB

Amber is a 6.7 billion parameter model from LLM360, an initiative dedicated to full transparency in large language model training. Every aspect of Amber's creation has been publicly documented and released, including the complete training data, all intermediate checkpoints, training code, and evaluation results. This level of openness makes Amber uniquely valuable for researchers studying training dynamics, data influence, and model behavior at scale. For local deployment, it offers solid general-purpose text generation at a size that fits comfortably on mid-range consumer GPUs, though users primarily seeking chat performance may prefer models specifically tuned for instruction following.

Mellum 4B Base

JetBrains · 4.0B · runs from 2.8 GB

Mellum 4B Base is a 4.0B-parameter open language model from JetBrains in the Mellum family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 160M

JackFram · 162M · runs from 0.4 GB

Llama 160M is a 162M-parameter open language model from JackFram in the Llama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Olmo 3 32B Think SFT

Allen AI · 32.2B · runs from 14.5 GB

Olmo 3 32B Think SFT is a 32.2B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 65,536 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Distilgpt2

distilbert · 88M · runs from 0.0 GB

DistilGPT-2 is a distilled version of OpenAI's GPT-2 model, compressed to just 88 million parameters while retaining much of the original model's text generation ability. Created using knowledge distillation techniques, it offers significantly faster inference than the full GPT-2 with only a modest reduction in output quality. This model is one of the lightest autoregressive language models available and can run on virtually any hardware, including CPUs. It is a practical choice for educational projects, quick prototyping, and applications where inference speed and minimal resource usage are more important than state-of-the-art generation quality.

Supra2 100M Instruct

SupraLabs · 101M · runs from 0.4 GB

Supra2 100M Instruct is a 101M-parameter open language model from SupraLabs. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gpt2 Mini

erwanf · 39M · runs from 0.0 GB

Gpt2 Mini is a 39M-parameter open language model from erwanf. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Falcon 7B

TII UAE · 7.2B · runs from 3.4 GB

Falcon 7B was one of the first truly competitive open-source large language models, released in mid-2023 by the Technology Innovation Institute in Abu Dhabi. Trained on the massive RefinedWeb dataset, it demonstrated that carefully curated web data could rival models trained on more traditionally assembled corpora. At 7 billion parameters, Falcon 7B helped establish the 7B class as the sweet spot for local inference, offering genuine language understanding on consumer GPUs with as little as 6 GB of VRAM.

Bloomz 560M

BigScience · 559M · runs from 0.3 GB

Bloomz 560M is a 559M-parameter open language model from BigScience. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Molmo2 4B

Allen AI · 4.9B · runs from 2.5 GB

Molmo2-4B is Allen Institute for AI's (Ai2) vision-language model for image, video, and multi-image understanding and grounding, built on a Qwen3-4B-Instruct backbone with a SigLIP 2 vision encoder. It is trained on Ai2's own curated Molmo2 datasets rather than third-party captioning data of unclear provenance, and beyond ordinary visual question answering it supports pointing at and tracking objects across video frames. The card reports state-of-the-art results among open weight-and-data models on short-video understanding, counting, and captioning, with competitive results on long videos. At under 5 billion parameters, it fits on a single consumer GPU. Context length is 36,864 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2025.

Pythia 14M

EleutherAI · 14M · runs from 0.0 GB

Pythia 14M is a 14M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

NousCoder 14B

Nous Research · 14.8B · runs from 6.9 GB

NousCoder 14B is a 14.8B-parameter open language model from Nous Research. It supports a context window of up to 81,920 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen 1 8B

Alibaba · 1.8B · runs from 0.9 GB

Qwen-1.8B is Alibaba's first-generation 1.8-billion-parameter base language model, pretrained from scratch on over 2.2 trillion tokens of Chinese, English, multilingual, code, and math data, with the same roughly 150,000-token vocabulary used across the Qwen family. It is a raw pretrained model rather than a chat assistant; Alibaba's aligned Qwen-1.8B-Chat is built on top of it. Its main selling point is low-cost deployment: the card reports int4/int8 quantized versions needing under 2GB of memory for inference and as little as 6GB for fine-tuning, so it runs comfortably on almost any consumer GPU or even a laptop. Context length is 8,192 tokens. It is released under a custom Tongyi Qianwen Research License, free for academic research, with commercial use requiring direct contact with Alibaba. It was published in November 2023.

Falcon 40B

TII UAE · 41.8B · runs from 19.6 GB

Falcon-40B is TII's 40-billion-parameter causal decoder-only base language model, pretrained from scratch on 1,000 billion tokens of the RefinedWeb dataset enhanced with curated corpora. At release TII described it as the best fully open model available, outperforming LLaMA, StableLM, and MPT on the Open LLM Leaderboard, with an inference-optimized architecture using FlashAttention and multi-query attention. It is a raw, pretrained model meant to be fine-tuned for chat or other downstream tasks rather than used directly; TII's own Falcon-40B-Instruct is the ready-to-use chat version built on top of it. Trained mainly on English, German, Spanish, and French, it needs a multi-GPU workstation in half precision, while a 4-bit quantization brings it within reach of a single high-memory GPU. License is Apache 2.0, permitting unrestricted commercial and research use. It was published in May 2023.

OLMo 1B HF

Allen AI · 1.2B · runs from 1.1 GB

OLMo 1B HF is a 1.2B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen 14B

Alibaba · 14.2B · runs from 6.6 GB

Qwen-14B is Alibaba's first-generation 14-billion-parameter base language model, pretrained from scratch on over 3 trillion tokens of Chinese, English, multilingual, code, and math data, with a roughly 150,000-token vocabulary for broader multilingual coverage. It is a raw pretrained Transformer, not tuned for conversation; Alibaba's aligned Qwen-14B-Chat is the assistant built on top of it. The card reports it beating other open models of similar size, and in some benchmarks larger models too, on Chinese and English evaluation suites. At 14B parameters it needs a capable consumer GPU at full precision, considerably less once quantized. Context length is 8,192 tokens, extendable further with the NTK-aware interpolation and window-attention techniques described in the card. It is released under a custom Tongyi Qianwen License Agreement that is free for research, with commercial use requiring a separate application to Alibaba. It was published in September 2023.

T5 Paraphrase Paws

Vamsi · 223M · runs from 0.1 GB

T5 Paraphrase Paws is a 223M-parameter open language model from Vamsi. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Phi 3 Mini 128k Instruct

Microsoft · 3.8B · runs from 2.7 GB

Phi 3 Mini 128k Instruct is a 3.8B-parameter open language model from Microsoft in the Phi 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Sarvam 30B Uncensored

aoxo · 32.2B · runs from 14 GB

Sarvam 30B Uncensored is a 32.2B-parameter open language model from aoxo. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 4 12B IT Assistant

Google · 12B · runs from 5.4 GB

Gemma 4 12B IT Assistant is a 12B-parameter open language model from Google in the Gemma 4 family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.