← News

ResearchSources: arXiv · 18 Nov 2026

Researchers show final-answer confidence alone cannot tell reasoning models when to stop

An arXiv paper argues that a model's verdict on its final answer cannot predict whether more thinking helps or hurts, and proposes measuring a commitment-verdict gap instead.

In plain words

Reasoning models can reach a correct answer, then change it by continuing to think. The authors argue that judging only the final answer cannot reveal whether stopping early would help. They propose comparing the answer the model would give now with the one it would end up with.

What was announced

The paper, posted on arXiv, studies reasoning models that hold a correct answer but can lose it by continuing their chain of thought. It argues that the verdict alone, which final-correctness probes estimate, cannot decide whether to stop, because as a forecast of a fixed outcome it has no predictable drift. The authors define a commitment-verdict gap that equals the expected remaining change in answer quality and the accuracy gained by stopping.

On MATH-500, the authors report that at 379 points where DeepSeek-R1-Distill-Qwen-1.5B held a correct answer, continuing lost 3.6 points on average, with the loss concentrated in a few problems. Across six models from three families, on MATH-500 and OlympiadBench with budgets up to 16,384 tokens, declining accuracy curves came from which chains remained running, while chains still reasoning kept or improved their answers. An oracle stopping rule gained at most 1.8 and 2.5 points at the 1.5B and 7B scales. The framework, the authors say, indicates what to measure before building a stopping rule.

What it does not show

  • The evidence is the paper's abstract only; the full article's methods and results were not reviewed, and the measurements are the authors' own without independent confirmation.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.