All LLM Models
Browse 45 LLM models with VRAM requirements, quantization options, and hardware compatibility.
Model List
NVIDIA Nemotron 3.5 Lightning 30B A3B BF16
NVIDIA · 31.6B · runs from 19.3 GB at Q4
1.1M
269NVIDIA Nemotron 3.5 Lightning 30B A3B BF16 is a mixture-of-experts chat model in NVIDIA's Nemotron line, with roughly 30 billion total parameters and about 3 billion active per token. That active-parameter count drives inference speed, while the larger total governs memory footprint, and the Lightning name points to a variant tuned for fast responses; weights are published in BF16 precision. The model offers a 256K token context window and is released under a custom license rather than a standard open-source one. With only about 3 billion parameters active per token, it is well suited to running on a single high-end consumer GPU, especially once quantized, despite its larger total parameter count.
NVIDIA Nemotron 3 Nano 30B A3B BF16
NVIDIA · 31.6B · runs from 19.3 GB at Q4
NVIDIA Nemotron 3 Nano 30B A3B is a mixture-of-experts model with 31.6 billion total parameters but only around 3 billion active per token, giving it the intelligence of a much larger model with the speed of a small one. This BF16 version preserves full precision for maximum output quality. The MoE architecture makes this model especially interesting for local deployment. You get reasoning and instruction-following capabilities that punch well above what a traditional 3B model can deliver, while inference stays fast because only a fraction of the network fires for each token.
NVIDIA Nemotron 3.5 Bolt E4B BF16
NVIDIA · 8.0B · runs from 5.6 GB at Q4
NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 is a small language model from NVIDIA, post-trained from Google's Gemma-4-E4B-it and released in full BF16 precision. It targets local voice and device assistants, interactive personas, and embedded interfaces, with better multi-turn tool use, instruction following, and input grounding than the base model. It has about 8 billion parameters, 4 billion of them effective. A Q4 build needs roughly 5 GB of memory, and NVIDIA also publishes a separate 4 GB Q4_K_M release, so it runs on a modest GPU or a laptop. The context length is up to 64K tokens for input and output each. It is released under the OpenMDW-1.1 license, and the card says it is ready for commercial or non-commercial use. It was published on October 5, 2026. Its card reports 63.79 on IFBench against 42.84 for Gemma 4 E4B-it, and 68.51 on BFCL-v3. Thinking can be switched on or off in the chat template, and vLLM is the documented serving path.
NVIDIA Nemotron Nano 9B v2
NVIDIA · 8.9B · runs from 6.0 GB at Q4
NVIDIA Nemotron Nano 9B v2 is a compact yet capable chat model from NVIDIA, packing 8.9 billion parameters into a size that runs comfortably on a wide range of consumer GPUs. Built on NVIDIA's Nemotron architecture, it delivers strong instruction-following and conversational performance while keeping VRAM requirements modest. This second-generation Nano model reflects NVIDIA's push to make high-quality language models accessible on local hardware. It's an excellent starting point for users who want a responsive, general-purpose assistant without needing top-tier GPU memory.
NVIDIA Nemotron 3 Nano 4B BF16
NVIDIA · 4.0B · runs from 2.9 GB at Q4
NVIDIA Nemotron 3 Nano 4B BF16 is a 4.0B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron Nano 12B v2
NVIDIA · 12.3B · runs from 11.0 GB at Q4
NVIDIA Nemotron Nano 12B v2 is a 12.3B-parameter open language model from NVIDIA in the Nemotron family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron Orchestrator 8B
NVIDIA · 8.2B · runs from 7.6 GB at Q4
Nemotron Orchestrator 8B is a 8.2B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 40,960 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron 3 Nano Omni 30B A3B Reasoning BF16
NVIDIA · 33.0B · runs from 25.6 GB at Q4
Nemotron 3 Nano Omni 30B A3B Reasoning BF16 is a 33.0B-parameter open language model from NVIDIA in the Nemotron family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron Labs Dory1.0 4B BF16
NVIDIA · 4.0B · runs from 5.7 GB at Q4
NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 is a 4 billion parameter looped model from NVIDIA Labs, built for general-purpose reasoning on a small footprint. It applies a shared block of layers repeatedly, so one checkpoint can trade inference compute for accuracy by changing the loop count, and it mixes sliding-window and full-attention layers. NVIDIA recommends two loops as the default. A Q4 build needs roughly 2.5 GB of memory, so it fits almost any GPU and many CPU-only machines. The context window is up to 256K tokens, and the model supports English. It is released under the OpenMDW-1.1 license and was published on October 16, 2026. Its card says last-loop KV caching avoids extra cache memory as loops are added, and that it was pre-trained natively in NVFP4 before continuing in BF16. It also lists BF16, MXFP8, NVFP4, and Q5_K_M formats. vLLM support is still in a pull request, and llama.cpp support requires building from a Dory-enabled source tree.
Nemotron Cascade 2 30B A3B
NVIDIA · 31.6B · runs from 19.3 GB at Q4
Nemotron Cascade 2 30B A3B is a 31.6B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Llama 3 3 Nemotron Super 49B V1 5
NVIDIA · 49.9B · runs from 37.2 GB at Q4
Llama 3.3 Nemotron Super 49B is a 49.9-billion parameter chat model by NVIDIA, built on a modified Llama 3.3 architecture. It occupies a unique size point between the common 70B and 8B tiers, offering strong reasoning and conversational ability while requiring less VRAM than full 70B models. NVIDIA's Nemotron Super training pipeline applies extensive alignment tuning to optimize helpfulness and factual accuracy. The model typically needs 32GB or more of VRAM for local inference at reduced precision, placing it within reach of high-end consumer GPUs like the RTX 4090 or professional workstation cards.
NVIDIA Nemotron Nano 9B v2 Japanese
NVIDIA · 8.9B · runs from 6.0 GB at Q4
NVIDIA Nemotron Nano 9B v2 Japanese is a specialized variant of the Nemotron Nano 9B v2, fine-tuned for Japanese language understanding and generation. At 8.9 billion parameters, it maintains the same hardware-friendly footprint as the English version while delivering natural Japanese conversational ability. For users looking to run a Japanese-language assistant locally, this model offers a rare combination of compact size and dedicated language optimization from a major hardware vendor. It handles Japanese text with the fluency you'd expect from a purpose-built model rather than a multilingual afterthought.
Cosmos Reason2 2B
NVIDIA · 2.4B · runs from 3.5 GB at Q4
Cosmos Reason2 2B is NVIDIA's 2.4-billion-parameter open reasoning vision-language model for physical AI, built on the Qwen3-VL-2B-Instruct backbone and designed to let robots and embodied agents perceive a scene and reason step by step about space, time, and physical common sense before acting. It generates long chains of thought and can output structured spatial data such as 2D/3D point localization, bounding boxes, and trajectories, plus OCR reading of scene text, going beyond a general chat assistant into physical-world planning. At just 2.4 billion parameters it is small enough to run on a single consumer GPU. Context length is 256,000 tokens, up sharply from the 16,000 tokens of the original Cosmos Reason1. It is released under the NVIDIA Open Model License, a custom license permitting commercial use and redistribution provided derivatives carry a "Built on NVIDIA Cosmos" attribution notice. It was published in December 2025, alongside a larger 8B sibling.
Llama 3 3 Nemotron Super 49B V1
NVIDIA · 49.9B · runs from 37.2 GB at Q4
Llama 3 3 Nemotron Super 49B V1 is a 49.9B-parameter open language model from NVIDIA in the Llama 3 family. It supports a context window of up to 131,072 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron Labs Audex 30B A3B
NVIDIA · 30B · runs from 23.5 GB at Q4
Nemotron Labs Audex 30B A3B is a 30B-parameter open language model from NVIDIA in the Nemotron family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Cosmos Reason2 8B
NVIDIA · 8.8B · runs from 8.4 GB at Q4
Cosmos Reason2-8B is NVIDIA's 8.8-billion-parameter open reasoning vision-language model for physical AI, built on a Qwen3-VL-8B-Instruct backbone and tuned to reason step by step about video and images the way a human would when planning actions in the real world. Rather than just labeling objects, it applies physics, spatio-temporal understanding, and common sense to tasks like robot planning, autonomous-vehicle video captioning, and video-analytics annotation, producing structured outputs such as 2D/3D point localization, bounding boxes, trajectory coordinates, and on-screen OCR text. It ships alongside a smaller 2B variant for edge deployment, while the 8B model needs a capable single GPU or more, less once quantized. Context length is roughly 256,000 tokens, up sharply from 16,000 tokens in the original Cosmos Reason 1. It is released under the NVIDIA Open Model License, a custom license that permits commercial use and derivative models but requires attribution ("Built on NVIDIA Cosmos") and prohibits removing its safety guardrails. It was published in December 2025.
Nemotron Mini 4B Instruct
NVIDIA · 4B · runs from 3.2 GB at Q4
Nemotron Mini 4B Instruct is a 4B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 4,096 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
LocateAnything 3B
NVIDIA · 3.8B · runs from 3.2 GB at Q4
LocateAnything-3B is NVIDIA's 3.8-billion-parameter vision-language model for visual grounding rather than open-ended chat: referring-expression grounding, dense multi-object detection, GUI element grounding, point-based localization, and document or OCR layout grounding. Its core contribution is Parallel Box Decoding, which predicts a complete bounding box in one parallel step instead of token-by-token autoregressive decoding, giving up to 2.5x higher throughput while preserving geometric consistency. It combines a Qwen2.5-3B-Instruct language backbone with a MoonViT-SO-400M vision encoder and was trained on 12 million images with over 138 million grounding queries; NVIDIA has folded it into the Nemotron 3 Nano Omni model for agentic and computer-use grounding. At under 4 billion parameters, it runs on a single consumer GPU. Context length is 32,768 tokens. It is released under NVIDIA's non-commercial research license, permitting academic and non-profit use only; commercial use requires a separate license from NVIDIA. It was published in May 2026.
Cosmos Reason2 32B
NVIDIA · 33.4B · runs from 23 GB at Q4
Cosmos Reason 2 32B is NVIDIA's 32-billion-parameter reasoning vision-language model for physical AI and robotics, built on Qwen3-VL-32B-Instruct. Given a text prompt and an image or video, it thinks through the answer using physics understanding and common sense, and the card describes uses such as robot planning, video analytics and dataset annotation, with improved spatio-temporal understanding and timestamp precision over the first generation. At this size it needs a high-end GPU with 24 GB or more of VRAM, or a unified-memory machine, once quantized. The context window is 262,144 tokens, and the card cites input of up to 256K tokens. It is released under the NVIDIA Open Model License, which the card describes as allowing commercial use and derivative models, with additional conditions; the repository uses gated access. Published in April 2026, it sits alongside 2B and 8B siblings in the Cosmos-Reason2 family.
NVIDIA Nemotron 3.5 Lightning 30B A3B Base BF16
NVIDIA · 31.6B · runs from 19.3 GB at Q4
NVIDIA Nemotron 3.5 Lightning 30B A3B Base BF16 is a 31.6B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron Labs Diffusion 3B
NVIDIA · 3.8B · runs from 3.9 GB at Q4
Nemotron Labs Diffusion 3B is a 3.8B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron Parse 2.0
NVIDIA · 903M · runs from 2.0 GB at Q4
NVIDIA Nemotron Parse 2.0 is a sub-1-billion-parameter vision-encoder-decoder model purpose-built for document parsing rather than open-ended chat: given a page image, it outputs structured text with layout classes, bounding boxes, and reading order for elements like titles, paragraphs, tables, charts, and footnotes. It pairs a ViT-H vision encoder based on NVIDIA's C-RADIO with a 10-block mBART decoder, and over its predecessor v1.2 it adds roughly 20,000 new vocabulary tokens for more efficient multilingual (especially CJK and Indic-script) OCR, a dedicated chart class for chart-to-table parsing, and stronger table detection and text extraction. At under a billion parameters it runs on a single modest GPU. License is the OpenMDW License Agreement version 1.1, permitting both commercial and non-commercial use; the bundled tokenizer is separately licensed under CC-BY-4.0. It was published in August 2026.
Llama 3.1 Nemotron Nano VL 8B V1
NVIDIA · 8.7B · runs from 8.4 GB at Q4
NVIDIA has released Llama 3.1 Nemotron Nano VL 8B, an 8-billion-parameter vision-language model focused on document intelligence. This system pairs a C-RADIOv2-H vision encoder with the Llama-3.1-8B-Instruct language model. Key use cases include image summarization, text-image analysis, optical character recognition, interactive question answering, and text chain-of-thought reasoning. The model supports single-image inference in English only; according to the card, a 4-bit AWQ build can run on Jetson Orin devices or laptops. At this size, it fits onto a single consumer GPU once quantization is applied. The specifications call for an input plus output budget of 16K tokens. Use follows the NVIDIA Open Model License, with the Llama 3.1 Community Model License also in effect; additionally, the card requires a "Built with Llama" notice. It was published in June 2025. NVIDIA's documentation points to Nemotron Nano v2 12B VL as the newer alternative.
Llama 3.1 Nemotron Nano 8B V1
NVIDIA · 8B · runs from 7.3 GB at Q4
Llama 3.1 Nemotron Nano 8B is an 8-billion parameter chat model by NVIDIA, a compact entry in the Nemotron family derived from Meta's Llama 3.1 architecture. It applies NVIDIA's alignment and fine-tuning techniques to deliver improved response quality over the base Llama 3.1 8B Instruct model at the same parameter count. The model runs on consumer GPUs with 8GB or more of VRAM and supports a 128K token context window. Its small footprint and NVIDIA-tuned quality make it a practical option for local inference on mainstream hardware.
NVIDIA Nemotron 3.5 Bolt E4B 4GB
NVIDIA · 4B · runs from 4.8 GB at Q4
NVIDIA Nemotron 3.5 Bolt E4B 4GB is a 4B-parameter open language model from NVIDIA in the Nemotron family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron H 8B Base 8K
NVIDIA · 8.1B · runs from 7.9 GB at Q4
Nemotron H 8B Base 8K is a 8.1B-parameter open language model from NVIDIA in the Nemotron family. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Mistral NeMo Minitron 8B Instruct
NVIDIA · 8.4B · runs from 6.7 GB at Q4
Mistral NeMo Minitron 8B Instruct is a 8.4B-parameter open language model from NVIDIA in the Mistral family. It supports a context window of up to 8,192 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
NVIDIA Nemotron 3 Nano 30B A3B Base BF16
NVIDIA · 31.6B · runs from 24.6 GB at Q4
NVIDIA Nemotron 3 Nano 30B A3B Base BF16 is the foundation model version of the Nemotron 3 Nano 30B, offered in full BF16 precision. Unlike the chat-tuned variants, this base model hasn't been instruction-tuned, making it suitable for fine-tuning, research, or custom alignment workflows. At 31.6 billion total parameters with a mixture-of-experts architecture, the base model gives developers and researchers a strong starting point for building specialized applications. It retains all the architectural benefits of the MoE design while leaving the behavioral layer open for customization.
Nemotron Cascade 8B
NVIDIA · 8B · runs from 7.5 GB at Q4
Nemotron Cascade 8B is a 8B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 32,768 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Nemotron Labs Diffusion 14B
NVIDIA · 13.5B · runs from 11.8 GB at Q4
Nemotron Labs Diffusion 14B is a 13.5B-parameter open language model from NVIDIA in the Nemotron family. It supports a context window of up to 262,144 tokens. See its VRAM requirements by quantization and which GPUs and Macs can run it locally below.
Understanding LLM VRAM Requirements
How much VRAM you need depends on the model size and quantization level. Quantization reduces the precision of model weights, trading small quality losses for significantly lower VRAM usage. For example, a 7B parameter model needs ~14 GB at FP16 but only ~4 GB at Q4_K_M quantization.