← News

Models & toolsSources: arXiv · 16 Nov 2026

Researchers propose KAD protocol to test how language models recover after adopting false answers

An arXiv paper introduces Knowledge Answer Divergence, an evaluation method that separates a model's first acceptance of a false claim from how it behaves afterward, including recovery and abstention.

In plain words

When a chatbot accepts a false correction, what it says next can depend on how the conversation continues. The proposed KAD method pauses the conversation at the moment of that first false answer and then tests different continuations from the same point. This lets researchers measure persistence, recovery and refusal separately instead of using one overall score.

What was announced

The paper, posted to arXiv, describes KAD as an event-conditioned evaluation protocol. It freezes a conversation at the first explicit adoption of a designated false target and branches independent probes from that exact history, which the authors say enables matched comparisons of post-adoption behavior and history-management interventions that a single susceptibility score cannot capture.

The reported evaluation covers eight local checkpoint conditions and six hosted endpoints, plus an audited 300-question rerun. In corrected Llama 3.2 Instruct 3B controls, a neutral-history branch produced 42 of 46 correct responses versus 16 of 46 under full history, and a cue-placement control changed false adoption by 27.3 percentage points in Qwen 3.5 4B. Twelve independently authored pressure templates showed substantial sensitivity to wording. The authors present these results as evidence that KAD can evaluate factual reliability beyond initial susceptibility.

What it does not show

  • The evidence is the paper's abstract only; the full article, methods and peer review status are not available in the supplied sources.
  • All reported numbers come from the authors' own evaluation; no independent confirmation is provided.
  • The abstract does not state which specific models beyond the named examples were included or how the hosted endpoints were configured.

For security and IT teams

Scope
The findings concern evaluation methodology for language model factual reliability and do not describe a deployed product or fix.

Sources

Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 17 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.