Videos

A ~85-second animated explainer of DiffusionGemma, Google's open-weight model that generates text the way image diffusion models generate pixels. Autoregressive LLMs emit one token per step, leaving the GPU memory-bound and mostly idle; DiffusionGemma instead opens a 256-token canvas seeded with random vocabulary noise and refines every position in parallel across several denoising passes — Uniform State Diffusion. Unlike earlier masked diffusion, where a filled-in [MASK] token is locked forever, low-confidence tokens can be re-noised and corrected in later passes. Under the hood one Gemma backbone toggles between two modes: a causal prefill mode that reads the prompt and writes the KV cache, and a bidirectional denoising mode where every canvas token attends to every other. For long-form text, completed canvases are appended to the KV cache and a fresh canvas is opened — autoregression at the block level, diffusion within each block. Press play on the featured player above.