Gemma 4 (deepmind.google/models/gemma/gemma-4/) is Google DeepMind’s flagship family of open-weight foundation models. Built on technology shared with Gemini 3 and released under a permissive Apache 2.0 license, Gemma 4 spans edge to workstation sizes—including Effective 2B (E2B), Effective 4B (E4B), 12B Unified, 26B A4B MoE, and 31B Dense. It introduces configurable step-by-step thinking modes, native function calling, multimodal perception (text, vision, and audio on smaller variants), and up to 256K token context windows optimized for local and on-device execution.

Running advanced reasoning, tool calling, and long-context multimodal inference on local developer hardware has historically been bottlenecked by high parameter sizes and memory footprint constraints. Gemma 4 bridges this gap with high intelligence-per-parameter scaling, utilizing hybrid local/global sliding window attention, per-layer embeddings (PLE) on edge models, and sparse mixture-of-experts (MoE) routing to achieve high throughput on phones, laptops, and single-GPU workstations.

Model Lineup & Architectural Overview

Model Variant	Total / Effective Parameters	Active Parameters	Context Window	Supported Modalities	Primary Target Hardware
Gemma 4 E2B	2.3B Effective (5.1B w/ embeddings)	2.3B	128,000 Tokens	Text, Image, Audio	Mobile (Android/iOS), Edge NPU
Gemma 4 E4B	4.5B Effective (8.0B w/ embeddings)	4.5B	128,000 Tokens	Text, Image, Audio	Consumer Laptops, ML Kit
Gemma 4 12B Unified	11.95B (Encoder-free architecture)	11.95B	256,000 Tokens	Text, Image, Audio	Mid-tier GPUs, Mac Apple Silicon
Gemma 4 26B A4B MoE	25.2B Sparse MoE (128 experts)	3.8B Active (8 routed + 1 shared)	256,000 Tokens	Text, Image	Fast single-GPU inference (vLLM / SGLang)
Gemma 4 31B Dense	30.7B Dense Parameters	30.7B	256,000 Tokens	Text, Image	Workstations & Multi-GPU Serving

Key Capabilities and Innovations

• Configurable Thinking & Extended Reasoning: Features an integrated thinking mode where the model reasons step-by-step before producing its response, delivering strong gains across STEM, code refactoring, and logical deduction benchmarks.

• Native Agentic Tool Use & Structured Output: Engineered for autonomous agent loops with native function calling, JSON schema enforcement, and first-class system prompt support (system role) without requiring fragile wrapper prompts.

• Per-Layer Embeddings (PLE) for Edge Efficiency: The smaller E2B and E4B tiers utilize per-layer embeddings, enabling compact compute footprints during execution while preserving representational capacity across 140+ languages.

• Encoder-Free Unified Modality Architecture (12B): The 12B Unified variant eliminates separate vision/audio encoders, projecting raw visual patches and audio waveforms directly into the decoder transformer space for lower latency and single-pass fine-tuning.

• Turnkey Open-Source Ecosystem Support: Available via Hugging Face, Kaggle, Ollama, LM Studio, vLLM, SGLang, LiteRT-LM, llama.cpp, and Google AI Studio with day-one support for INT4, FP8, and BF16 formats.