← News

ResearchSources: arXiv · 18 Nov 2026

GauntletScore matches top fact-checkers on LLM-AggreFact while logging its evidence

A new arXiv abstract describes an evidence engine that reaches 79.51 balanced accuracy on the full LLM-AggreFact test split, level with the best published single-model checkers while recording the reasoning behind each verdict.

In plain words

Fact-checkers are tools that verify whether an AI-written sentence is actually backed by its source document. GauntletScore asks independent reader models from different providers and combines their answers with a fixed statistical rule, keeping a record of every step. Its authors report it matches the best published checkers on a standard benchmark.

What was announced

The engine puts typed questions to independent readers from different providers and records each answer as a measurement in an evidence ledger. A fixed Bayesian rule then combines the measurements, counting a contradiction only when two readers agree. Its few rules and thresholds were set on the benchmark's development split, and the test split was run once on an engine frozen before the first test claim, under a stopping plan committed in advance.

On the full LLM-AggreFact test split of 29,320 claims across 11 datasets, the authors report 79.51 balanced accuracy, above every average published on the leaderboard as of 15 November 2026 and level with the best single model, nimble-v4-max, whose published score of 79.45 falls inside the engine's 95% interval of 78.47 to 80.53. The engine beats two of its three readers used alone by 1.66 and 2.17 points, though without the AggreFact-CNN dataset those leads shrink to 0.90 and 1.32. Its largest margin is on full-length news articles checked against multi-sentence summaries, where it scores 76.45 against 70.1 for the next best. The authors release per-claim predictions and the scoring script.

What it does not show

  • The evidence is the paper's abstract only; the full article was not reviewed, so the method and results could not be independently checked.
  • The comparison to nimble-v4-max relies on that model's published dataset scores, not a rerun by the authors, and the leaderboard snapshot is dated 15 November 2026.
  • The benchmark covers the 11 datasets in LLM-AggreFact; performance on other claim types or documents is not established.

For security and IT teams

Scope
The reported results cover the LLM-AggreFact test split of 29,320 claims across 11 datasets, with the strongest margin on news articles checked against multi-sentence summaries.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.