DiffusionGemma: When Text Generation Stops Being Sequential

Google DeepMind's DiffusionGemma (diffusiongemma-26B-A4B-it on Hugging Face) is an open-weight model that generates text the way image diffusion models generate pixels — not one token at a time, but as a whole block that gets progressively refined. It's built on the Gemma 4 26B-A4B Mixture-of-Experts backbone (roughly 25.2B total / 3.8B active parameters), with a diffusion head bolted onto that same architecture rather than a model designed from scratch.

Denoising instead of predicting

A standard autoregressive LLM emits tokens one after another, each conditioned on everything before it — which means one small matrix multiplication per token, and a GPU that's mostly waiting on memory rather than actually computing. DiffusionGemma instead starts with a block of up to 256 masked placeholder tokens and runs several iterative denoising steps over the whole block at once, using bidirectional attention so tokens can inform their neighbors both forward and backward. Confident tokens resolve first and help disambiguate the ones still uncertain, so a passage "snaps into focus" over a handful of passes rather than being committed to word by word.

In practice that means the model can denoise 15–20 tokens' worth of output per forward pass instead of one — shifting the bottleneck from memory bandwidth to raw compute. A nice side effect: because tokens get revisited across denoising steps rather than locked in immediately, the model has more room to self-correct mid-generation than a strictly left-to-right decoder does.

Where it actually pays off: local, single-user inference

The compute-bound framing matters most for exactly the case that's usually the worst for autoregressive models: batch size one, a single person waiting on a response. NVIDIA's RTX AI Garage optimized DiffusionGemma for GeForce RTX and RTX PRO cards as well as DGX Spark and DGX Station, and the throughput numbers back up the pitch — reportedly over 1,000 tokens/sec on a single H100 in FP8, around 150 tokens/sec on DGX Spark, and up to 2,000 tokens/sec on DGX Station, roughly 4x an equivalent autoregressive model in that same single-user regime. NVIDIA recommends at least 16GB of VRAM to comfortably hold the model's active parameters, and ships an NVFP4-quantized variant for lighter local setups.

Faster, not smarter

The honest caveat, repeated across independent write-ups: DiffusionGemma trails standard autoregressive Gemma 4 on established benchmarks like MMLU and coding evals. This isn't a capability upgrade — it's a different point on the latency/throughput curve, aimed squarely at making local generation feel instant rather than making the model better at reasoning. With a 256K context window, native function calling, multilingual support across 35+ languages, and multimodal input under an Apache 2.0 license, it's a genuinely useful tradeoff for the right workload — just not a replacement for reaching for the strongest model when the task calls for it.

Update — July 25, 2026: the same bet, now in production

When this post went up in June, DiffusionGemma read as an interesting one-off. Six weeks later it's clear it was neither the first nor the last — it's one entry in a category that's been building for over a year.

Worth correcting the record on the "first" part, because I under-credited it above. Inception Labs shipped Mercury Coder in February 2025 — the world's first commercial-scale diffusion LLM, fifteen months before DiffusionGemma. It came out of Stefano Ermon's Stanford lab, which had co-authored the score-based generative modelling work that image diffusion rests on. Mercury Coder Mini and Small hit 1,109 and 737 tokens/sec on a single H100, up to 10× speed-optimized autoregressive models at comparable quality. Inception generalized it to open-ended chat in June 2025 and published a technical report. It has since shipped Mercury 2 — the first reasoning-capable dLLM, at 1,009 tokens/sec and 1.7 s end-to-end against 14.4 s for Gemini 3 Flash — and Mercury Edit 2, a model built solely for next-edit prediction in an IDE.

That last one is the most instructive product in the category, because it picks the task diffusion is actually best at. Next-edit prediction is latency-critical — a suggestion that arrives after you've already typed the line is worthless — and the outputs are short and structurally constrained. Exactly the profile where parallel denoising pays and none of its long-horizon reasoning weaknesses get exercised. It scores 75.6% quality at 221 ms, ahead of Claude 4.5 Haiku (71.4%) and GPT-5.4 Nano (73.5%). Google shipped an open-weight generalist to see what diffusion could do; Inception shipped a narrow closed product where the answer was already known.

So Google didn't open the category. What Google did was make it inspectable — which is what the rest of this update depends on.

Celeris Labs announced celeris-1 on July 24 — and the framing is notably different from Google's. DiffusionGemma is an open-weight research release you download and run; celeris-1 is a commercial inference product, diffusion decoding behind an OpenAI-compatible API (point your existing SDK at api.celeris.ai and keep your code). The company's pitch is that they've built "a new inference architecture for language models, built on diffusion, that achieves latency and quality previous diffusion systems could not."

Their published benchmark table makes the tradeoff unusually legible:

Read it as a frontier — up and to the left wins, more accurate and faster — and three things stand out.

Against Gemini 2.5 Flash and Mercury 2, celeris-1 simply dominates: better accuracy and lower latency, no trade at all. The Mercury 2 comparison is the meaningful one, because Mercury is the other diffusion model and the only entry in the same latency class — celeris-1 beats it by 12 points at 100 ms less. That's the head-to-head that actually tests the "we solved diffusion quality" claim, and celeris-1 wins it.

Against GPT-5 and Gemini 3.5 Flash Lite, it's a genuine trade: they're 6–7 points more accurate, and 13–16× slower. Whether that's a good deal is entirely a question of what you're building. For a voice agent, 158 ms versus 2 seconds is the difference between conversation and waiting; for a research assistant, seven points of MMLU-Pro matter more than either.

Credit where it's due on methodology: that timing basis column is more disclosure than most vendor benchmark pages offer. It also shows the comparison is mixed — the Gemini entries are measured end-to-end including network, the rest are server-reported, which excludes it. And their accompanying chart specifies "reasoning budget 0: the fastest configuration for each model," with Mercury 2 in instant mode. That's a defensible way to compare latency floors, but it means the autoregressive models are benchmarked with reasoning off, which is not how anyone runs GPT-5 when they care about the answer. All figures are self-reported.

What DiffusionGemma's fuller disclosure reveals

Here's where having both models' numbers side by side earns its keep. Google published fifteen benchmarks for DiffusionGemma against its own same-size autoregressive sibling — Gemma 4 26B A4B, same backbone, same lab. That's a controlled experiment on what diffusion decoding actually costs, and the answer is: it depends enormously on the task.

The pattern is coherent. Broad knowledge recall survives parallel denoising nearly intact. What degrades is multi-step reasoning (AIME loses 19 points), agentic tool use (Tau2, 82% retained), long-context retrieval, and precise structured output — OmniDocBench's edit distance more than doubles, because getting exact document structure right is precisely what "revise the whole block at once" is worst at.

Which puts celeris-1's single published number in perspective: MMLU-Pro is one of diffusion's two strongest events on that entire table. That doesn't make 75.9% wrong or dishonest — it's a real score on a real benchmark. But it's the benchmark most flattering to the architecture, and celeris-1 is explicitly pitched at agents and real-time applications, which is the category where DiffusionGemma gives up closer to 18%. A single MMLU-Pro figure can't tell you whether celeris-1 shares that weakness or has engineered around it. The honest position is that we don't know yet.

The wider field has filled in fast. Ant Group's LLaDA2.2-flash took diffusion decoding after agentic work — adding edit operations so parallel decoding can insert and delete, not just substitute, which is what makes tool calls and JSON survive the process. Read against the table above, that's a direct engineering answer to exactly the structured-output and agentic weaknesses DiffusionGemma exposed.

The result that complicates "faster, not smarter"

Then there's NVIDIA's Nemotron-Labs-Diffusion, which refuses the premise. Instead of choosing between architectures, it trains one set of weights on a joint autoregressive and diffusion objective, then switches modes at inference by changing only the attention mask — causal for AR, bidirectional for diffusion. That enables a third mode that falls out of the design: the model drafts in diffusion mode and verifies in autoregressive mode, over a shared KV cache. Speculative decoding, with no separate draft model.

The 8B base results are the most interesting numbers in this whole post:

Reference points from the same table: Qwen3-8B (AR) 71.58 · LLaDA-8B (pure diffusion) 54.92 · Dream-7B (pure diffusion) 65.30

Read that carefully, because it undercuts the framing this post has used since June. The model's diffusion mode slightly beats its own autoregressive mode — and its self-speculation mode beats both, at 4.7× the tokens per forward. No accuracy tax at all. The pattern repeats at the other scales: at 3B and 14B, pure diffusion mode does cost a little accuracy (−2.6 and −1.0 points), but self-speculation recovers it and comes out ahead of AR (+0.3 and +0.7), while decoding 5–7× more tokens per forward.

Meanwhile the pure diffusion baselines in that same comparison pay dearly — LLaDA-8B at 54.92 against Qwen3-8B's 71.58 is a 17-point gap, and Nemotron's own diffusion mode beats LLaDA by 17.2 points and Dream-7B by 6.8.

So the honest conclusion is narrower than "diffusion is faster but dumber." The accuracy tax appears to be the price of committing to diffusion as the only mode — of throwing away the left-to-right linguistic prior that autoregression provides. Use diffusion to draft and autoregression to verify, and on this evidence you keep the speed and give up nothing. Which is what DiffusionGemma's uneven benchmark profile was hinting at all along: the tasks where it collapses are the ones most dependent on strict sequential structure.

One caveat on comparability: tokens-per-forward is an architectural efficiency measure, not wall-clock latency. celeris-1's 158 ms is an end-to-end product number; Nemotron's 7× is a throughput property that still has to survive a real serving stack. And these are the paper's own figures on its own suite.

NVIDIA made the same argument a second way, a month later. Nemotron-Labs-TwoTower — a genuinely different model, frequently confused with the tri-mode one — splits the work between a frozen autoregressive context tower that processes clean tokens causally, and a trainable diffusion denoiser tower that refines noisy blocks by cross-attending to it. Bolted onto the existing Nemotron-3-Nano-30B-A3B backbone, it keeps 98.7% of the autoregressive baseline's quality at 2.42× wall-clock throughput. Where tri-mode trains one weight set on both objectives from the start, TwoTower is the cheap retrofit: take a pretrained AR model you already have, freeze it, and attach parallel decoding to the side.

Two different routes, same conclusion — you don't have to give up autoregression to get diffusion's speed.

Which answers the question this update opened with. "Is diffusion decoding viable?" was settled a while ago — Inception has been selling it since February 2025, and there are now implementations from Google, NVIDIA, Ant Group and Celeris besides. The better question was whether diffusion stays a separate class of model you pick for latency, or becomes a decoding mode any model can switch into — and NVIDIA's two results are the strongest evidence yet for the second. celeris-1 and DiffusionGemma are bets on diffusion as an architecture. Nemotron is a bet that it's a gear.