DFlash: Diffusion Drafts the Block

The EAGLE line spent three papers raising α. Look at where that leaves the speedup formula, though: with acceptance lengths of 4–5 already achieved, the remaining cost is the drafting loop itself. An EAGLE head still runs sequentially — γ tokens means γ forward passes of the head, plus tree bookkeeping, all on the latency-critical path. In the chapter 1 arithmetic: α is nearly maxed; the frontier is $c$.

DFlash’s answer: draft the entire block in a single forward pass — with a small diffusion language model.

Why a Diffusion Drafter

Diffusion LLMs generate by iteratively refining a block of masked tokens in parallel — all positions at once — rather than left to right. As standalone generators they still trail autoregressive models in quality, which kept them out of production. DFlash’s reframe: in a speculative pipeline, draft quality only sets the acceptance rate. The lossless acceptance rule from chapter 1 means a diffusion model’s weaknesses cost speed, never correctness — while its parallelism attacks exactly the term ($c$) that the autoregressive drafters cannot. It is the rare case where a technology’s flaw is priced at zero and its strength at full value.

So each cycle becomes: one parallel draft pass proposes the block → one target pass verifies it. Two big matmuls, no token-by-token loop on either side.

KV Injection: EAGLE’s Trick, Upgraded

A drafter that never reads the context can’t guess well, and re-encoding the context with the draft model would burn the savings. DFlash conditions the drafter through KV injection: the target model’s hidden representations of the context are passed through the draft model’s KV projections and written directly into the draft model’s KV cache, at every draft layer. The drafter never builds its own understanding of the prompt — it drafts inside the target’s.

This is the same load-bearing idea as EAGLE’s feature conditioning, taken further: not one feature vector per step, but the target’s full processed context, available to every drafted position in parallel.

The Model at a Glance

What it shows. One DFlash cycle. The target (left) has already processed the context; its hidden states at every layer are pushed through the draft model’s KV projections and written straight into the draft KV cache (amber arrows) — the drafter never builds its own view of the prompt. The draft diffusion model (right) starts from a fully masked K-token block, runs its layers over all K positions at once, and repeats for S refinement passes (S = 1–2 in practice), committing positions in confidence order. The finished block goes back to the target for a single verify pass (green) under the chapter-1 rule. Count the big matmuls per cycle: two — one draft, one verify. Compare EAGLE’s feature-flow diagram: same KV-reuse instinct, but there the draft head loops γ times.

Watch the Block Get Drafted

What it models. A controlled drafting race. Every cycle, both drafters must produce the same K-token block and both blocks face the same acceptance coins — so the accepted tokens are identical by construction, and the only variable left is drafting time. The amber lane types its draft one head-pass per token; the blue lane starts from a fully masked block (▒▒▒, pulsing) and commits positions over S refinement passes in confidence order — watch the middle of the block resolve before its left edge, something a left-to-right drafter cannot do.

Knobs. K is the block size, S the number of refinement passes (real block-diffusion drafters use 1–2), α the shared draft quality. A target verify pass is fixed at 12 tick-equivalents.

Try this. At the defaults (K = 8, S = 2), the gap chip reads (K+12)/(S+12) ≈ 1.43× — DFlash’s entire edge, earned while accepting the same tokens. Now push K to 16: the amber lane’s drafting phase becomes an ever-longer crawl while the blue lane barely notices (S didn’t change) and the gap grows to 2×. This is the measured table below in miniature: identical acceptance lengths, very different clocks. The one thing this toy hides is that a longer parallel block accepts less per position — suffix decay — which is the next page’s problem.

What It Buys

The revealing comparison is acceptance length versus speedup (SGLang measurements, EAGLE-3 with a 5-layer head as baseline):

Task	EAGLE-3 acc-len / speedup	DFlash acc-len / speedup
GSM8K	4.2 / 2.1×	4.2 / 3.3×
HumanEval	4.3 / 2.2×	4.0 / 3.2×
MT-Bench	3.1 / 1.4×	3.0 / 2.2×

Read that table twice: DFlash accepts the same number of tokens per pass — and is ~1.5× faster anyway. The entire gain is the drafting cost collapsing from γ sequential steps to one parallel pass. Overall the paper reports >6× lossless acceleration across models (up to 2.5× over EAGLE-3), >4.3× throughput on a 397B MoE, and NVIDIA reports up to 15× for gpt-oss-120b on Blackwell at matched interactivity — parallel drafting composes multiplicatively with hardware that has compute to spare.

The systems side: Spec V2

Half of production speculative-decoding time can disappear into host–device synchronization rather than math. SGLang’s second-generation speculation engine (“Spec V2”) overlaps host-side bookkeeping and KV allocation with GPU execution, worth another ~33% on top of the algorithm. Algorithmic and systems gains stack — a theme this site’s cache track readers will find familiar.

Common misconception: “A diffusion drafter makes the output slightly ‘diffusion-flavored’.”

Nothing the drafter proposes survives without clearing the target’s acceptance rule, position by position. The output distribution is the target’s, exactly — same theorem as chapter 1, no exceptions for exotic drafters. If the diffusion block is bad, you’ll see it in the drafts wasted metric, not in the text.

Block drafting does introduce a new failure mode of its own, and the architecture diagram already contains it: what a block position attends to at its left neighbor is that neighbor’s [MASK] placeholder, never the token the drafter is about to sample there. All $K$ logit vectors are therefore finished before the first token of the block is drawn. The later positions are guessed with the least information, so acceptance decays along the block — and under batch load, verifying doomed suffixes wastes real capacity. Why the block decays — the draft samples marginals, not the joint — and what it costs to fix are DSpark’s job, next; where block drafting sits against the MTP lineage (DeepSeek-V3’s $D=1$ self-drafter) is the closing frame.

References

• DFlash: Block Diffusion for Flash Speculative Decoding. ICML 2026. arXiv:2602.06036 · code
• LMSYS. The Next Generation of Speculative Decoding: DFlash and Spec V2. lmsys.org
• NVIDIA. Boost Inference Performance up to 15× on Blackwell Using DFlash. developer.nvidia.com