Can I Run LLM Model Locally?

Qwen3 Coder 30B A3B Instruct

Alibaba · 30.5B · runs from 8.8 GB

564.9K 1.3K

Qwen3 Coder 30B A3B Instruct is a code-specialized Mixture of Experts (MoE) model from Alibaba Cloud's Qwen 3 Coder series, with 30 billion total parameters and approximately 3 billion active parameters per forward pass. The MoE architecture allows it to deliver strong coding performance while keeping per-token compute costs low, making it faster at inference than comparably capable dense models. The model is instruction-tuned for programming assistance, code generation, debugging, and software engineering conversation. It requires VRAM proportional to its total 30B parameter count for loading weights, but benefits from efficient inference throughput due to its low active parameter count. Released under the Apache 2.0 license.

Chat</>Code

Qwen3.6 35B A3B

Alibaba · 36.0B · runs from 15.7 GB

3.2M 2.9K

Qwen3.6 35B A3B pairs a mixture-of-experts design with roughly 35 billion total parameters, only about 3 billion of which are active per token. That small active footprint keeps generation speed close to a 3B-class dense model, even though the full parameter set must still be held in memory, similar to a dense model of that size, meaning local use still calls for a single high-end consumer GPU once quantized. The model also accepts image input alongside text, extending it to visual question answering and image-grounded chat. It supports a 256K token context window for long documents and extended conversations, and is released under the Apache 2.0 license. Published in April 2026, it gives Qwen 3.6 users a faster-inference alternative to the dense Qwen3.6 27B model released the same generation.

Vision

Qwen2.5 7B Instruct

Alibaba · 7.6B · runs from 2.7 GB

9.7M 2.2K

Qwen2.5 7B Instruct is a 7.6-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 2.5 series. It supports a 128K token context window and is fine-tuned for conversational AI, instruction following, and general assistant tasks. Its efficient size makes it well-suited for local deployment on consumer GPUs with 8GB or more of VRAM. The model delivers strong performance for its parameter class across reasoning, multilingual understanding, and coding tasks. It benefits from the improved pretraining data and techniques of the Qwen 2.5 generation. Released under the Apache 2.0 license and widely supported by inference frameworks such as llama.cpp, vLLM, and Ollama.

Chat

Qwen3 8B

Alibaba · 8.2B · runs from 2.9 GB

12.7M 2.0K

Qwen3 8B is an 8.2-billion parameter instruction-tuned model from Alibaba Cloud's Qwen 3 series. It is a general-purpose chat model that delivers strong performance across reasoning, multilingual understanding, and coding tasks while remaining efficient enough to run on consumer GPUs with 8GB or more of VRAM. Like other Qwen 3 models, it supports hybrid thinking mode for flexible reasoning depth. The model benefits from the improved pretraining data and training methodology of the Qwen 3 generation, offering notable quality gains over Qwen 2.5 at the same parameter count. It is widely supported by inference frameworks including llama.cpp, vLLM, and Ollama. Released under the Apache 2.0 license.

Chat

Qwen3.8 27B

Alibaba · 27.8B · runs from 12.6 GB

6.9M 16.1K

Alibaba's Qwen3.8 27B is a dense, vision-capable model with around 28 billion parameters, continuing the Qwen line into its 3.8 release. It can process images alongside text prompts, supporting tasks like visual question answering and image-grounded chat in addition to general-purpose reasoning and coding assistance. At this size, the model fits on a single high-end consumer GPU once quantized, putting it within reach of enthusiast local setups rather than requiring server-class hardware. It ships with a 256K token context window, enough for long-form documents or extended multi-turn sessions, and is distributed under the Apache 2.0 license. Released in August 2026, it represents one of the more recent entries in Alibaba's Qwen 3.8 family of open-weight models.

Vision

Gemma 4 E4B IT

Google · 8.0B · runs from 3.2 GB

4.4M 1.6K

Gemma 4 E4B IT packs Google's Gemma 4 architecture into a compact, roughly 8-billion-parameter footprint, positioned as the mid-sized option in Gemma 4's efficiency-focused E-series alongside the smaller E2B variant. It is tuned for chat and instruction-following rather than multimodal input, focusing on dialogue quality within a lightweight package. Parameter counts in this range are well suited to local inference on a single consumer GPU, even at higher precision, and comfortably so once quantized. The model provides a 128K token context window, sufficient for most chat and document-assistance use cases without needing the largest Gemma 4 variants. It is released under the Apache 2.0 license, and was published on March 2, 2026, shortly before the larger dense and mixture-of-experts Gemma 4 models that followed.

Chat