FailureGen reuses recorded robot failures to generate annotated synthetic training videos
A new arXiv paper describes a four-stage pipeline that transplants motion from real failure recordings into new contexts, producing synthetic videos with event annotations for robot learning research.
In plain words
Robots fail in ways that are hard to film at scale, so datasets of failures stay small. FailureGen takes motion from recorded failures and re-renders it in other settings using a world model, a generative video system conditioned on robot actions. The result is synthetic failure footage with annotations that can train models to detect failures.
What was announced
The authors say robot failures are difficult to collect at scale, which limits the failure scenarios represented in robot learning datasets. Their method fine-tunes an action-conditioned Cosmos world model on successful and failed real executions, then annotates source recordings to find failure-associated donor segments and compatible successful hosts. Donor motion is transferred into host contexts, with optional spatial offsets, to create continuous video rollouts, and generated samples are annotated and filtered by observed events and visual quality.
The paper reports two implementations with different tasks and robot configurations, with visual failure detection as one downstream application. In a preliminary comparison, a detector trained on real and generated data reached an average precision of 0.51 versus 0.41 with real data alone, detecting 33 of 36 failure intervals compared with 28 of 36. The authors position the work as expanding data for studying robot failures and building downstream learning methods.
What it does not show
- The evidence is limited to the paper's abstract; the full article's methods, experiments and caveats are not covered here.
- The detector comparison is described by the authors as preliminary, and no independent confirmation of the results is available.
For security and IT teams
- Scope
- The published evidence covers the paper's abstract only, describing the method and headline results without implementation or availability details.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.