← News

Agents & MCPSources: arXiv · 20 Nov 2026

FinVeriQA prototype routes financial questions to specialized agents, but validation misses swapped-company errors

An open-source multi-agent research prototype for answering questions about SEC filings and market data shows strong routing and computation results, yet a live case study exposed evidence-verification gaps.

In plain words

The researchers built a test system where different small AI models handle separate jobs: one picks the right tool for each question, Python does all math, and another model writes a cited answer that software then checks. It worked well in tests, but in one live run the checker approved answers that described one company's data under another company's name.

What was announced

The FinVeriQA pipeline, described in a paper by its authors on arXiv, assigns each step to a component suited to it: a compact typed-decision model called Laya routes questions and screens out advice requests, Python performs every calculation, retrieval and tools fetch evidence, Mistral-7B writes a cited answer, and a code validator confirms that cited identifiers exist and that numbers match computed notes. In live experiments the authors report Laya routed better than Mistral-7B (0.85 versus 0.52), Mistral-7B computed 29% of 100 financial calculations correctly against 100% for Python, and a live quote cut price error from 210% to 0.00%.

The same paper reports a cautionary result: in a live case study, both agent answers passed validation while describing AMD sample data under NVIDIA's and Alphabet's names. A controlled study on the FinRank benchmark offers a possible explanation. When one figure in shared evidence was corrupted, self-consistency stayed confident while accuracy fell, and a shared-evidence check of the kind the validator performs caught 35% of errors, while checking against the original passage caught 77%. The authors conclude that verification must be independent of the evidence the writer saw.

What it does not show

  • The source is the paper's abstract only, so experimental setup, dataset details and full results cannot be verified from the supplied text.
  • The live case study involved both agent answers describing AMD sample data under other companies' names, but the abstract does not establish how often such errors occur overall.
  • Reported figures are the authors' own measurements; no independent confirmation is provided.

For security and IT teams

Scope
FinVeriQA is described as an open-source multi-agent research prototype over SEC filings and live market data, exposed through an MCP server, not a production system.

Sources

Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 21 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.