← News

Models & toolsSources: arXiv · 19 Nov 2026

Infilling attacks pull more training data from diffusion language models

A new extraction protocol shows that probing masked positions reveals memorization in diffusion LLMs that prefix-only tests miss, including redacted personal data.

In plain words

Diffusion language models generate text by filling in masked tokens anywhere in a sequence, unlike standard models that only continue from a prefix. The paper's authors show attackers can exploit this by choosing which positions to mask, recovering far more training data, including personal information, than prefix-based probing.

What was announced

The arXiv paper introduces infilling extraction, a protocol that parameterizes extraction with an arbitrary binary mask rather than restricting probes to a prefix. Testing LLaDA-8B and Dream-7B across five extraction modes, three training pipelines, and three corpora, the authors report that mask geometry governs extractability: edge-conditioned masks recovered up to three times more verbatim sequences than prefix-conditioned ones, and bidirectional access opened extraction channels unavailable to autoregressive models.

The most concrete risk involves personal information. According to the paper, an adversary holding a PII-redacted copy of training data can use targeted infilling to raise PII recall on fine-tuned diffusion models by 2.5-4.5x over prefix-conditioned extraction, narrowing the gap to a scale-matched autoregressive model and, for LLaDA email addresses under greedy prefix-conditioned extraction, exceeding it. The authors also note that decoding parameters measurably affect extraction, while a follow-up supervised fine-tuning stage does not erase prior memorization.

What it does not show

  • Evidence is limited to the paper's abstract; full experimental setup, baselines, and threat-model assumptions are not available for verification.
  • Findings are reported by the paper's authors and have not been independently confirmed.
  • Extraction results cover two named models and specific corpora; applicability to other diffusion LLMs is not established in the supplied evidence.

For security and IT teams

Scope
The reported extraction results apply to diffusion language models such as LLaDA-8B and Dream-7B, not to autoregressive models, which the paper says prefix-only probing was designed for.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.