← News

Agents & MCPSources: arXiv · 18 Nov 2026

Study says noisy evaluation, not weak proposals, stalls self-improving AI agents

An arXiv paper argues agent evolution is bottlenecked when evaluations cannot reliably tell better artifacts from worse ones, so selection errors accumulate across rounds.

In plain words

Self-improving AI agents work by proposing changes, such as new prompts or skills, and keeping the ones that score higher in tests. The paper's authors argue the real bottleneck is not generating good changes but that noisy test scores make it hard to know which changes are actually better. As a result, bad changes can be kept and good ones discarded.

What was announced

The authors, writing in an arXiv abstract, describe agent evolution as a loop in which a proposer writes candidate artifacts such as prompts, skills and memories, and selection accepts those that score higher. They call their explanation the Resolution Hypothesis: evolution stalls not because methods fail to propose better artifacts, but because noisy evaluations cannot reliably identify them. The claim rests on experiments spanning seven evolution methods and six models, involving more than 10 million LLM calls.

The paper reports support for the hypothesis at two levels. At the candidate level, helpful artifacts exist, but selection performs no better than random at distinguishing them from harmful ones, so errors accumulate across rounds. At the method level, typical margins of a few percentage points between methods are smaller than the difference between two independent runs of the same method, which the authors say means published rankings often rest on luck. They conclude the field should focus less on designing more candidate-generation systems and more on making true improvements detectable and accumulable.

What it does not show

  • The evidence is the paper's abstract only; the full methodology, task details and peer review status are not available in the supplied source.
  • The findings are the authors' own reported results and have not been independently confirmed.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.