Retriever-Perceiver-Reasoner framework targets multimodal scientific document reasoning
A research team describes an agentic system that retrieves evidence from text, figures and tables, organizes it, and answers questions, trained through diagnostic-guided co-evolution.
In plain words
Answering questions about scientific papers can require combining information from text, figures and tables. The authors propose a three-part agent: one part picks relevant evidence, another organizes it for the question, and a third writes a grounded answer. The parts are trained together so errors in finding or using evidence can be addressed separately.
What was announced
The paper, posted on arXiv, argues that retrieval alone can miss complementary support, and that answer-level feedback alone does not show whether a mistake came from acquiring evidence or from using it. The framework addresses this with three components: a Retriever that sequentially selects source-aligned evidence under a context budget, a Perceiver that builds a question-conditioned latent ontology organizing the selected multimodal evidence in a shared continuous space, and a Reasoner that combines this latent guidance with the original sources to produce grounded answers.
The three components are trained through what the authors call diagnostic-guided co-evolution: alternating updates optimize them with a shared answering reward, while two signals, Retrieval Potential Feedback and Latent Efficacy Feedback, identify questions needing better retrieval or latent guidance and direct targeted resampling. On four benchmarks and seven evaluation settings, the authors report average scores of 69.9 and 75.0 with 7B and 8B Reasoner backbones, exceeding the strongest baseline by 4.0 and 9.1 points respectively. These results are self-reported in the abstract and have not been independently verified.
What it does not show
- The evidence is limited to the paper's abstract; full method details, experimental setup and baselines are not covered by the supplied passage.
- Reported scores and baseline comparisons come from the authors and are not independently confirmed.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.