← News

Agents & MCPSources: arXiv · 19 Nov 2026

RoboBuilder benchmark tests whether AI coding agents can design robots that survive the real world

Researchers introduced RoboBuilder, a benchmark where coding agents design robot hardware and software from standard parts, then see their designs physically built and tested on a real obstacle course.

In plain words

A research team created a challenge in which AI coding agents design both a robot's body and its control program from standard parts, motors and one custom 3D-printed piece. Designs that pass in simulation are actually built and run on a physical course. The benchmark measures how close AI agents are to engineering machines that work outside the simulator.

What was announced

The gap between simulation and reality proved significant. Under public part data, none of 15 designs that passed in simulation managed to cross the real course, according to the paper. To address this, an agent learns the simulator from measurements it requests and from videos of real runs. The benchmark's scale was substantial: across four frontier coding agents, ten obstacles at three difficulty levels, the study reports 16,427 simulations covering 12,774 distinct robots and 135 physical robots built in an estimated 240 hours.

Results were mixed by difficulty. Agents solved nearly all easy obstacles and invented mechanisms that the authors' search baselines did not reach, but they faltered at harder levels, when a single robot had to cross every obstacle, and in the real world, where 66% of simulated passes held up across all levels. Closing the loop helped: shown a video of its real-world failure, an agent redesigned the robot, sometimes correcting its own simulator, and removed 72% of level-1 and level-2 failures. The authors frame RoboBuilder as a measure of progress toward agents that engineer real machines.

What it does not show

  • The evidence is the paper's abstract only; full methodology, baselines and evaluation details are not available in the supplied excerpt.
  • Results come from the benchmark authors' own testing; no independent confirmation is provided.
  • The abstract does not name the four coding agents or specify which designs succeeded at which difficulty levels.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.