Study finds LLM agents act on stale stored evidence even when current facts are present
A controlled arXiv study of Qwen3 checkpoints reports that persistent-memory agents can follow outdated stored information despite current authoritative evidence, with effects varying by model size and cue type.
In plain words
Some AI agents keep a memory of past information and can keep acting on it after it becomes outdated. Researchers tested this with controlled tasks and found the agents often trusted the old stored note even when current, correct information was placed in front of them. The effect depended on cues like how fresh the note appeared, and was not fixed across model sizes.
What was announced
The arXiv paper, whose abstract describes two controlled, action-scored test suites, reports that reliance on stale stored values reached 0.92-1.00 across the tested Qwen3 checkpoints in a Benefit suite where stored information was needed. In a Safety suite where current evidence was supplied as static prompt text, larger checkpoints resisted a plainly stale note but suffered substantial harm when the stale note appeared newer, according to the authors.
Using a full factorial design manipulating label, recency, source authority and position, the authors report stronger recency effects in the larger tested checkpoints and a position-effect sign reversal within Qwen3. Metadata improved accuracy at all four checkpoints, with larger gains at 4B and 8B, and pre-resolution of stored values yielded near-ceiling accuracy. The authors state the results characterize cue-dependent vulnerability across tested checkpoints without isolating capability or evaluating a complete memory lifecycle; RGB and MisBench corroborate stale-value reliance, and a framing control showed the failure extends beyond memory-labeled content.
What it does not show
- The evidence is the paper's abstract only; the full article's methods and results were not reviewed, and the authors note the study does not isolate capability or evaluate a complete memory lifecycle.
For security and IT teams
- Scope
- Findings apply to the tested Qwen3 0.6/1.7/4/8B checkpoints in two controlled suites, with Llama-Instruct comparisons and RGB and MisBench corroboration, not to all agent systems.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.