Researchers introduced RoboBuilder, a benchmark where coding agents design both a robot’s hardware and its control program using standard parts and a custom 3D-printed piece. Designs that pass simulation are built and tested on a physical obstacle course.
In the researchers’ tests, none of 15 initial designs that passed simulation completed the real course. Across four coding agents, 135 robots were built; simulated successes carried over to real trials 66% of the time across all difficulty levels.
When shown videos of real-world failures, agents redesigned robots and sometimes corrected their simulator, removing 72% of failures on the two easier levels. They still struggled with harder obstacles and with designing one robot that could cross every obstacle.
Why it matters
For robotics researchers using simulation to screen designs, RoboBuilder shows that a passing simulation is not enough: in these tests, about one in three simulated passes failed to hold up across the real-world trials.