Coherence boosting: an inference-time fix for long-context attention in language models
Researchers report that large language models underweight distant words when predicting the next token, and describe an inference procedure called coherence boosting that sharpens attention to long context without extra training.
In plain words
Language models sometimes lose track of meaning across long stretches of text. The authors say this happens because the models have not fully learned how far-away words affect the next word choice. Their fix, coherence boosting, adjusts the model at inference time — when it is running, not while training — and needs no additional training.
What was announced
In an arXiv paper abstract, the authors argue that long-range semantic coherence remains a challenge for automatic language generation and understanding, and that large language models have insufficiently learned the effect of distant words on next-token prediction. Coherence boosting is presented as an inference procedure that increases a language model's focus on a long context, meaning it changes how a trained model processes input rather than altering its weights.
The reported benefits come from two kinds of evaluation. Distributional analyses of generated ordinary text and dialog responses are said to show gains with pretrained models, and the authors report that applying coherence boosting to state-of-the-art models on various zero-shot NLP tasks yields performance improvements with no additional training. The abstract does not name the models, tasks, or metrics involved, so the size and generality of the gains cannot be verified from this source.
What it does not show
- The source is only the paper's abstract; it does not name the models, tasks, or metrics, and the reported gains have not been independently confirmed.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.