Google DeepMind's DiffusionGemma (diffusiongemma-26B-A4B-it) is an open-weight model that generates text the way image diffusion models generate pixels — not one token at a time, but as a whole block that gets progressively refined. It's built on the Gemma 4 26B-A4B Mixture-of-Experts backbone (roughly 25.2B total / 3.8B active parameters), with a diffusion head bolted onto that same architecture rather than a model designed from scratch.
Denoising instead of predicting
A standard autoregressive LLM emits tokens one after another, each conditioned on everything before it — which means one small matrix multiplication per token, and a GPU that's mostly waiting on memory rather than actually computing. DiffusionGemma instead starts with a block of up to 256 masked placeholder tokens and runs several iterative denoising steps over the whole block at once, using bidirectional attention so tokens can inform their neighbors both forward and backward. Confident tokens resolve first and help disambiguate the ones still uncertain, so a passage "snaps into focus" over a handful of passes rather than being committed to word by word.
In practice that means the model can denoise 15–20 tokens' worth of output per forward pass instead of one — shifting the bottleneck from memory bandwidth to raw compute. A nice side effect: because tokens get revisited across denoising steps rather than locked in immediately, the model has more room to self-correct mid-generation than a strictly left-to-right decoder does.
Where it actually pays off: local, single-user inference
The compute-bound framing matters most for exactly the case that's usually the worst for autoregressive models: batch size one, a single person waiting on a response. NVIDIA's RTX AI Garage optimized DiffusionGemma for GeForce RTX and RTX PRO cards as well as DGX Spark and DGX Station, and the throughput numbers back up the pitch — reportedly over 1,000 tokens/sec on a single H100 in FP8, around 150 tokens/sec on DGX Spark, and up to 2,000 tokens/sec on DGX Station, roughly 4x an equivalent autoregressive model in that same single-user regime. NVIDIA recommends at least 16GB of VRAM to comfortably hold the model's active parameters, and ships an NVFP4-quantized variant for lighter local setups.
Faster, not smarter
The honest caveat, repeated across independent write-ups: DiffusionGemma trails standard autoregressive Gemma 4 on established benchmarks like MMLU and coding evals. This isn't a capability upgrade — it's a different point on the latency/throughput curve, aimed squarely at making local generation feel instant rather than making the model better at reasoning. With a 256K context window, native function calling, multilingual support across 35+ languages, and multimodal input under an Apache 2.0 license, it's a genuinely useful tradeoff for the right workload — just not a replacement for reaching for the strongest model when the task calls for it.
Links
- Documentation: DiffusionGemma — ai.google.dev
- Architecture explainer: How DiffusionGemma works — ai.google.dev
- Weights: google/diffusiongemma-26B-A4B-it on Hugging Face
- Local inference: NVIDIA RTX AI Garage — Gemma Diffusion