Discrete Stochastic Localization lets one diffusion model serve several samplers
A new continuous-state framework aims to close the gap between continuous diffusion and masked discrete models, letting a single trained network support multiple sampling methods without extra training.
In plain words
Diffusion models generate text by gradually turning noise into tokens. The authors propose a way to do this with continuous states instead of masked discrete tokens, so one trained denoising network can sample in several different modes. In their tests, fine-tuning an existing masked model with this method improved generation quality across step budgets.
What was announced
Continuous diffusion has generally trailed masked discrete diffusion models (MDMs) on discrete sequence generation, according to the paper's abstract. The authors introduce Discrete Stochastic Localization (DSL), a continuous-state framework pairing a stochastic localization channel with unit-sphere token embeddings. They report this yields a Bayes-optimal denoiser that depends only on the noisy state, with no additional timestep conditioning.
Because a single trained denoising network supports multiple samplers, the same checkpoint can generate via continuous denoising, masked diffusion, random-order autoregressive generation, or a hybrid continuous-then-discrete route. The authors argue parallel generation here need not mean synchronized denoising: some tokens can fix context early while others stay open to refinement. Fine-tuning a pretrained MDLM checkpoint with DSL reportedly improved MAUVE generation-quality scores on OpenWebText across step budgets from T=128 to T=1024, and the same checkpoint supported random-order autoregressive sampling plus a hybrid sampler at as few as T=48 total steps, without distillation or further training.
What it does not show
- Evidence is limited to the paper's abstract; full experimental details, baselines and comparisons are not available in the supplied passages.
- Quality claims rest on the authors' own MAUVE measurements on OpenWebText and have not been independently confirmed.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.