Researchers propose evidence governance to stop medical AI agents being misled by faulty tool outputs
An arXiv paper shows tool-augmented medical agents can be flipped from correct to wrong answers by silent tool failures, and introduces a framework that filters tool outputs before the reasoner sees them.
In plain words
Some medical AI systems call external tools, such as classifiers or databases, and treat every tool answer as trustworthy evidence. The paper's authors show that when a tool fails quietly, the answering model can be pushed from a correct answer to a wrong one. Their fix is a separate checking step that decides which tool outputs count as evidence before the answering model sees them.
What was announced
The authors, writing in an arXiv abstract, describe the problem as "evidence hijacking": tool-augmented medical agents treat whatever their tools return as evidence, but tools can fail silently, and a frozen reasoner cannot ignore what enters its context. They report that cases the system would have answered correctly are flipped to wrong once tool outputs are added. Using an information-theoretic framing, they argue the reasoner loses the option to ignore tool output, so an output can carry negative usable information, and moving the decision about what to keep outside the reasoner restores that option.
As a response, the paper proposes context-independent evidence governance (CIEG). Each tool output is written as a card that can be checked against the raw input; an independent checker, seeing only the input and the cards, decides which cards become evidence; and the reasoner answers from admitted cards alone, with no training anywhere. On medical question answering, fundus glaucoma screening, and chest X-ray pneumonia classification, the authors report the highest accuracy among models of comparable scale, outperforming tool-using agents and multi-agent systems built on the same backbone at lower cost. These results come from the paper's own abstract and have not been independently confirmed.
What it does not show
- The source is only the paper's abstract, so the method details and reported results cannot be verified against the full article.
- Accuracy and cost claims are the authors' own and are not independently confirmed.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.