← News

Agents & MCPSources: arXiv · 18 Nov 2026

Study finds coding agents write strong test coverage but translation remains unreliable

An arXiv paper benchmarks three popular coding agents on writing unit tests and translating them between programming languages, finding uneven quality.

In plain words

Coding agents are AI tools that can write and run software tasks on their own. Researchers tested how well three of these agents write unit tests, which are small checks that confirm code works, and how well they convert those tests to other programming languages. The agents beat humans on coverage but struggled on larger projects and on translation.

What was announced

The paper, posted on arXiv, examines whether coding assistants built on large language models can be trusted to write reliable tests and to translate those tests between programming languages. The authors benchmarked three popular coding agents, each running with a native frontier model at high reasoning effort, and verified results across a broad suite of metrics.

The findings are mixed. Agent-written test suites far exceed human-written suites on traditional measures of coverage, but the authors attribute much of that advantage to the agents simply writing more tests. Quality degrades sharply as project size grows, and test translation by agents, while appearing strong in aggregate, remains inconsistent and unreliable according to the study. The work matters because much of the trust placed in coding agents rests on tests the agents themselves produce.

What it does not show

  • The source is an abstract only, so detailed metrics, agent names, and methodology are not available in the supplied passages.
  • The abstract does not name the three agents or models evaluated, and no independent replication of the findings is provided.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.