Reasoning helps models resist poisoned RAG context, arXiv study finds
A new arXiv paper reports that enabling deliberative reasoning in two large language models lowered how often detected misinformation in retrieved documents still shaped answers.
In plain words
Retrieval-augmented generation (RAG) lets a language model pull in outside documents, and attackers can plant misinformation in those documents. The paper finds that a model can notice the false content yet still repeat it. Letting the models reason step by step reduced that gap in tests, though it also made detection less frequent.
What was announced
The study, published as an arXiv abstract, examines knowledge-poisoning attacks, in which misinformation injected into retrieved documents influences model outputs. The authors argue that attack success and poison detection alone do not show whether detected poison keeps shaping the final answer, so they introduce two measures. Cordon Rate captures the probability that a model detects poison yet still produces a poison-aligned answer, while Leakage Rate measures how often poisoned context influences the answer even when the model is explicitly told to ignore retrieved documents.
Testing 200 SciFact questions with reasoning-disabled and reasoning-enabled configurations of DeepSeek-V4-Flash and Qwen3.6-Plus, the authors report that reasoning cut DeepSeek-V4-Flash's Cordon Rate from 0.205 to 0.075 and its Leakage Rate from 0.235 to 0.140. The same configuration, however, raised the Attack Success Rate from 0.233 to 0.298 and lowered the Poison Detection Rate from 0.965 to 0.665, and Qwen3.6-Plus showed the same qualitative pattern. The authors conclude that poison detection and downstream influence are distinct aspects of poisoning behavior, meaning a model that flags poisoned content may still echo it.
What it does not show
- The evidence is an abstract only; full methodology, experimental settings and peer review status are not visible in the supplied passage.
- Results cover two models on 200 SciFact questions, so findings may not extend to other models, datasets or attack types.
- The reported trade-off, with reasoning lowering detection rates while reducing poison influence, is stated by the authors and not independently confirmed.
For security and IT teams
- Scope
- The findings apply to RAG poisoning behavior of DeepSeek-V4-Flash and Qwen3.6-Plus on SciFact questions as reported in the paper's abstract, not to RAG systems generally.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.