All LLM Models
Browse 32 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Model List
Mistral Nemo Base 2407
Mistral AI · 12.2B · runs from 11.0 GB at Q4
60.1K
371Mistral-Nemo-Base-2407 is a 12-billion-parameter pretrained base language model jointly developed by Mistral AI and NVIDIA, intended as a drop-in replacement for the earlier Mistral 7B rather than a chat assistant; it has not been instruction-tuned or aligned, and the card notes it carries no built-in moderation mechanisms. It uses a standard transformer architecture with grouped-query attention and SwiGLU activations, and was trained on a large share of multilingual and code data across nine languages. At 12 billion parameters it fits comfortably on a single consumer GPU once quantized. Context length is 131,072 tokens (a 128k window). It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in July 2024, alongside an instruction-tuned Nemo variant.
Mixtral 8x22B v0.1
Mistral AI · 140.6B · runs from 88.4 GB at Q4
Mixtral-8x22B-v0.1 is Mistral AI's large sparse mixture-of-experts base model, a pretrained checkpoint with no instruction tuning and no built-in moderation, intended as the foundation for fine-tuned or instruct derivatives rather than direct chat use. It routes each token through 2 of 8 experts, giving roughly 39 billion active parameters out of about 141 billion total, so its per-token compute is far lighter than its total size at the cost of having to hold every expert in memory. Its pretraining data covers English, French, German, Spanish, and Italian. Because all experts must be resident even though only a fraction activate per token, it still needs a multi-GPU workstation even when quantized. Context length is 65,536 tokens. It is released under the Apache 2.0 license, permitting unrestricted commercial and research use, and was published in April 2024, ahead of an instruction-tuned Mixtral-8x22B-Instruct release.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.