Infilling attacks pull more training data from diffusion language models
A new extraction protocol shows that probing masked positions reveals memorization in diffusion LLMs that prefix-only tests miss, including redacted personal data.
In plain words
Diffusion language models generate text by filling in masked tokens anywhere in a sequence, unlike standard models that only continue from a prefix. The paper's authors show attackers can exploit this by choosing which positions to mask, recovering far more training data, including personal information, than prefix-based probing.
What was announced
The arXiv paper introduces infilling extraction, a protocol that parameterizes extraction with an arbitrary binary mask rather than restricting probes to a prefix. Testing LLaDA-8B and Dream-7B across five extraction modes, three training pipelines, and three corpora, the authors report that mask geometry governs extractability: edge-conditioned masks recovered up to three times more verbatim sequences than prefix-conditioned ones, and bidirectional access opened extraction channels unavailable to autoregressive models.
The most concrete risk involves personal information. According to the paper, an adversary holding a PII-redacted copy of training data can use targeted infilling to raise PII recall on fine-tuned diffusion models by 2.5-4.5x over prefix-conditioned extraction, narrowing the gap to a scale-matched autoregressive model and, for LLaDA email addresses under greedy prefix-conditioned extraction, exceeding it. The authors also note that decoding parameters measurably affect extraction, while a follow-up supervised fine-tuning stage does not erase prior memorization.
What it does not show
- Evidence is limited to the paper's abstract; full experimental setup, baselines, and threat-model assumptions are not available for verification.
- Findings are reported by the paper's authors and have not been independently confirmed.
- Extraction results cover two named models and specific corpora; applicability to other diffusion LLMs is not established in the supplied evidence.
For security and IT teams
- Scope
- The reported extraction results apply to diffusion language models such as LLaDA-8B and Dream-7B, not to autoregressive models, which the paper says prefix-only probing was designed for.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.