Reward model targets motion loss in instruction-guided video editing
arXiv preprint reports that video-editing reward models often prefer clips with reduced motion, and proposes a motion-aware reward trained against such failures.
In plain words
Video editors follow text instructions, and reward models score their output clips to guide training. The authors found these scores can rise when motion is removed, so a model may reward a nearly frozen video. Their fix adds a motion-sensitive input and trains against weakened-motion examples.
What was announced
The preprint, posted on arXiv, describes a failure the authors call motion collapse: reward models for instruction-guided video editing can favor temporal consistency while motion fades. Using controlled motion-suppression variants of real edits, they report that a composite reward on VE-Bench preferred the motion-suppressed candidate in 97% of cases, while learned video rewards and a VLM judge also struggled with mild suppression. Latent optimization and reward-guided fine-tuning were said to exploit this weakness, raising reward as motion declined, and a linear probe of the base reward backbone performed near chance on mild suppression.
As a remedy, the authors propose a motion-aware reward model that pairs an RGB frame-difference stream with preference training against motion-suppressed and fabricated-motion negatives, plus a distillation anchor to keep alignment with human quality judgments. They report that on the VE-Bench motion ladder the model selected motion-suppressed candidates 0% of the time, reached a Spearman correlation of 0.57 with human quality ratings, detected unseen suppression variants, and preserved motion during reward-guided fine-tuning on two editing pipelines.
What it does not show
- Evidence is limited to the paper's abstract; the full methodology, baselines and experimental details were not available for review.
- All figures, including the 97% and 0% rates and the 0.57 correlation, are author-reported on VE-Bench and have not been independently confirmed.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.