All LLM Models

Browse 982 LLM models with VRAM requirements, quantization options, and hardware compatibility.

Featured only

Understanding LLM VRAM Requirements

How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.

Manaca 1B Base

menezesbruno · 1.7B · runs from 1.1 GB

Manaca 1B Base is a 1.7B-parameter open language model from menezesbruno. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen 7B

Alibaba · 7.7B · runs from 3.6 GB

Qwen-7B is Alibaba's original 7.7-billion-parameter base language model from the first Qwen series, pretrained (not instruction-tuned) on over 2.4 trillion tokens of Chinese, English, multilingual, code, and math text, with a roughly 150,000-token vocabulary aimed at multilingual friendliness. Alibaba positions it as a foundation for further fine-tuning; the aligned chat assistant Qwen-7B-Chat is a separate release built on top of it. Its native training context is 8,192 tokens, and the card documents NTK-aware interpolation, LogN attention scaling, and windowed attention to extrapolate usable context out to 32,768 tokens. At under 8 billion parameters, it fits on a single consumer GPU. Context length is 8,192 tokens natively, extendable to 32,768 tokens with the NTK, LogN, and window-attention techniques above. It is released under the Tongyi Qianwen License Agreement, which allows academic use freely but requires applying for a commercial license for commercial deployment. It was published in August 2023, an early entry in a Qwen lineage now several generations ahead.

Salamandra 7B Instruct

BSC-LT · 7.8B · runs from 3.8 GB

Salamandra 7B Instruct is a 7.8-billion-parameter multilingual model developed by the Barcelona Supercomputing Center (BSC-LT) as part of a European initiative to build high-quality open language models. It has particular strength in Iberian languages including Spanish, Catalan, Portuguese, and Basque, while also supporting English and other major European languages. This model is an excellent choice for users who need strong performance in Spanish or other Iberian languages that are often underserved by mainstream LLMs. Running it locally ensures data privacy for sensitive multilingual workflows, and at 7B parameters it fits comfortably on a single consumer GPU with 8 GB or more of VRAM.

Qwen2.5 Math 7B

Alibaba · 7.6B · runs from 3.6 GB

Qwen2.5 Math 7B is a 7.6B-parameter open language model from Alibaba in the Qwen 2.5 family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Molmo2 8B

Allen AI · 8.7B · runs from 4.3 GB

Molmo2 8B is an 8.7-billion-parameter vision-language model from the Allen Institute for AI (Ai2), built on a Qwen3-8B backbone paired with a SigLIP2 vision encoder. It is tuned for image, video, and multi-image understanding, with particular strength in pointing, counting, and tracking objects across frames rather than just describing them. Its size suits local deployment on a mainstream-to-high-end consumer GPU once quantized, a step down from the 72B Molmo model it succeeds. The model supports a 36,864 token context window, adequate for multi-image prompts and moderate video rather than long documents. It is released under the Apache 2.0 license, allowing unrestricted commercial and research use. Published in December 2025, Molmo2 8B is notable for pixel-level grounding and for tracking objects over time in video.

Qwen1.5 7B

Alibaba · 7.7B · runs from 4.7 GB

Qwen1.5-7B is Alibaba's dense 7.7-billion-parameter base language model, part of the Qwen1.5 series that served as a beta step between the original Qwen and Qwen2. It is a decoder-only Transformer using SwiGLU activation, released as a pretrained checkpoint rather than a chat model; its card explicitly recommends applying further post-training such as SFT or RLHF rather than prompting it directly for text generation. It offers a stable 32K token context window and is distributed under Alibaba's Tongyi Qianwen license, a custom license rather than a fully open one. At 7.7 billion parameters, it runs comfortably on an 8-12 GB consumer GPU once quantized to 4-bit, making it an accessible option for local fine-tuning experiments.

PowerMoE 3B

ibm-research · 3.4B · runs from 1.7 GB

PowerMoE-3B is IBM Research's 3.4-billion-parameter sparse Mixture-of-Experts language model, built for general text generation, reasoning, and code rather than a specific vision or audio modality. It routes each token to 8 of 40 experts, activating roughly 800 million parameters per token according to IBM, keeping decoding fast even though the full set of experts still has to be held in memory. At this size it fits comfortably on a single mainstream consumer GPU once quantized. Context length is limited to 4,096 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in August 2024. Its defining feature is the Power learning-rate scheduler IBM used to train it, which the team reports lets an 800-million-active-parameter model match dense models with roughly twice as many active parameters.

LocoTrainer 4B

LocoreMind · 4.0B · runs from 2.2 GB

LocoTrainer 4B is a 4.0B-parameter open language model from LocoreMind. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

ChatCodeFunctions

Prometheus 7B V2.0

prometheus-eval · 7.2B · runs from 3.6 GB

Prometheus 7B V2.0 is a specialized judge model trained by prometheus-eval to evaluate the quality of outputs from other language models. At 7.2 billion parameters, it is designed to score and critique LLM responses against custom rubrics, making it a valuable tool for automated evaluation pipelines and benchmarking. Unlike general-purpose chat models, Prometheus is purpose-built for assessment tasks. It can provide structured feedback on dimensions like helpfulness, accuracy, and coherence. Useful for researchers, developers building LLM applications, and anyone who needs consistent automated evaluation without relying on paid API calls to frontier models. Runs comfortably on most modern GPUs with 8 GB or more of VRAM.

ReaderLM v2

jinaai · 1.5B · runs from 1.0 GB

ReaderLM v2 is a 1.5B-parameter open language model from jinaai. It supports a context window of up to 512,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Pebble 10M Chat

basically-ai · 11M · runs from 0.0 GB

Pebble 10M Chat is a 11M-parameter open language model from basically-ai. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

CodeQwen1.5 7B

Alibaba · 7.3B · runs from 3.5 GB

CodeQwen1.5-7B is Alibaba's 7.3-billion-parameter code-specialized base language model, built on the Qwen1.5 architecture and pretrained on 3 trillion tokens of code data covering 92 programming languages. It uses group-query attention for efficient inference and is a raw pretrained model, not tuned for chat; Alibaba advises against using it directly for conversation and instead for fine-tuning, code infilling, and code generation. The card highlights strong text-to-SQL and bug-fixing performance for its size. At 7B parameters it fits comfortably on a single consumer GPU, even less once quantized. Context length is 65,536 tokens. It is released under a custom Tongyi Qianwen Research License, free for academic research, with commercial use requiring direct contact with Alibaba. It was published in April 2024.

Qwen1.5 7B Chat

Alibaba · 7.7B · runs from 4.7 GB

Qwen1.5 7B Chat is a 7.7B-parameter open language model from Alibaba in the Qwen family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gemma 3 4B T1 IT

twinkle-ai · 3.9B · runs from 2.3 GB

Gemma 3 4B T1 IT is a 3.9B-parameter open language model from twinkle-ai in the Gemma 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

SmolLM2 360M

Hugging Face · 362M · runs from 0.5 GB

SmolLM2 360M is a 362M-parameter open language model from Hugging Face in the SmolLM family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Amber

LLM360 · 6.7B · runs from 3.2 GB

Amber is a 6.7 billion parameter model from LLM360, an initiative dedicated to full transparency in large language model training. Every aspect of Amber's creation has been publicly documented and released, including the complete training data, all intermediate checkpoints, training code, and evaluation results. This level of openness makes Amber uniquely valuable for researchers studying training dynamics, data influence, and model behavior at scale. For local deployment, it offers solid general-purpose text generation at a size that fits comfortably on mid-range consumer GPUs, though users primarily seeking chat performance may prefer models specifically tuned for instruction following.

Mellum 4B Base

JetBrains · 4.0B · runs from 2.8 GB

Mellum 4B Base is a 4.0B-parameter open language model from JetBrains in the Mellum family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Llama 160M

JackFram · 162M · runs from 0.4 GB

Llama 160M is a 162M-parameter open language model from JackFram in the Llama family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Distilgpt2

distilbert · 88M · runs from 0.0 GB

DistilGPT-2 is a distilled version of OpenAI's GPT-2 model, compressed to just 88 million parameters while retaining much of the original model's text generation ability. Created using knowledge distillation techniques, it offers significantly faster inference than the full GPT-2 with only a modest reduction in output quality. This model is one of the lightest autoregressive language models available and can run on virtually any hardware, including CPUs. It is a practical choice for educational projects, quick prototyping, and applications where inference speed and minimal resource usage are more important than state-of-the-art generation quality.

Supra2 100M Instruct

SupraLabs · 101M · runs from 0.4 GB

Supra2 100M Instruct is a 101M-parameter open language model from SupraLabs. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Gpt2 Mini

erwanf · 39M · runs from 0.0 GB

Gpt2 Mini is a 39M-parameter open language model from erwanf. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Falcon 7B

TII UAE · 7.2B · runs from 3.4 GB

Falcon 7B was one of the first truly competitive open-source large language models, released in mid-2023 by the Technology Innovation Institute in Abu Dhabi. Trained on the massive RefinedWeb dataset, it demonstrated that carefully curated web data could rival models trained on more traditionally assembled corpora. At 7 billion parameters, Falcon 7B helped establish the 7B class as the sweet spot for local inference, offering genuine language understanding on consumer GPUs with as little as 6 GB of VRAM.

Bloomz 560M

BigScience · 559M · runs from 0.3 GB

Bloomz 560M is a 559M-parameter open language model from BigScience. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Molmo2 4B

Allen AI · 4.9B · runs from 2.5 GB

Molmo2-4B is Allen Institute for AI's (Ai2) vision-language model for image, video, and multi-image understanding and grounding, built on a Qwen3-4B-Instruct backbone with a SigLIP 2 vision encoder. It is trained on Ai2's own curated Molmo2 datasets rather than third-party captioning data of unclear provenance, and beyond ordinary visual question answering it supports pointing at and tracking objects across video frames. The card reports state-of-the-art results among open weight-and-data models on short-video understanding, counting, and captioning, with competitive results on long videos. At under 5 billion parameters, it fits on a single consumer GPU. Context length is 36,864 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in December 2025.

Pythia 14M

EleutherAI · 14M · runs from 0.0 GB

Pythia 14M is a 14M-parameter open language model from EleutherAI. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

NousCoder 14B

Nous Research · 14.8B · runs from 6.9 GB

NousCoder 14B is a 14.8B-parameter open language model from Nous Research. It supports a context window of up to 81,920 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen 1 8B

Alibaba · 1.8B · runs from 0.9 GB

Qwen-1.8B is Alibaba's first-generation 1.8-billion-parameter base language model, pretrained from scratch on over 2.2 trillion tokens of Chinese, English, multilingual, code, and math data, with the same roughly 150,000-token vocabulary used across the Qwen family. It is a raw pretrained model rather than a chat assistant; Alibaba's aligned Qwen-1.8B-Chat is built on top of it. Its main selling point is low-cost deployment: the card reports int4/int8 quantized versions needing under 2GB of memory for inference and as little as 6GB for fine-tuning, so it runs comfortably on almost any consumer GPU or even a laptop. Context length is 8,192 tokens. It is released under a custom Tongyi Qianwen Research License, free for academic research, with commercial use requiring direct contact with Alibaba. It was published in November 2023.

OLMo 1B HF

Allen AI · 1.2B · runs from 1.1 GB

OLMo 1B HF is a 1.2B-parameter open language model from Allen AI in the OLMo family. It supports a context window of up to 2,048 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.

Qwen 14B

Alibaba · 14.2B · runs from 6.6 GB

Qwen-14B is Alibaba's first-generation 14-billion-parameter base language model, pretrained from scratch on over 3 trillion tokens of Chinese, English, multilingual, code, and math data, with a roughly 150,000-token vocabulary for broader multilingual coverage. It is a raw pretrained Transformer, not tuned for conversation; Alibaba's aligned Qwen-14B-Chat is the assistant built on top of it. The card reports it beating other open models of similar size, and in some benchmarks larger models too, on Chinese and English evaluation suites. At 14B parameters it needs a capable consumer GPU at full precision, considerably less once quantized. Context length is 8,192 tokens, extendable further with the NTK-aware interpolation and window-attention techniques described in the card. It is released under a custom Tongyi Qianwen License Agreement that is free for research, with commercial use requiring a separate application to Alibaba. It was published in September 2023.

T5 Paraphrase Paws

Vamsi · 223M · runs from 0.1 GB

T5 Paraphrase Paws is a 223M-parameter open language model from Vamsi. It supports a context window of up to 512 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.