← News

ResearchSources: arXiv · 20 Nov 2026

CoachSI uses a frozen vision-language model to coach robot policies in language

The arXiv paper describes a self-improvement method where a vision-language model names the robot's next step, enabling fine-tuning that lifts unassisted success on long-horizon tasks.

In plain words

Robots can perform small, single actions well but struggle to chain them into long tasks. CoachSI has a vision-language model watch the robot and say what the next step should be, in words the robot already understands. Successful coached runs are then used to retrain the robot so it can later manage on its own.

What was announced

The authors argue that generalist robot policies execute atomic skills reliably but fail when those skills must be composed into long-horizon tasks, and that self-improvement from the policy's own experience does not close the gap because such experience stays within the policy's existing distribution. Their answer is a frozen, off-the-shelf vision-language model that watches the robot and names the next step in the format of the policy's atomic instructions, without ever outputting an action.

Successful coached rollouts are relabeled in hindsight with the original task instruction and used to fine-tune the policy, which is then deployed unassisted. The paper reports that naming the next step raises subgoal completion by 4.6x over the unchanged task instruction on states where a World Action Model and a Vision-Language-Action model have stalled. After fine-tuning, unassisted success on long-horizon tasks in unseen scenes rises from 47.4% to 59.3%, which the authors state is 9.8 points above fine-tuning on human demonstrations and 7.3 points above fine-tuning on self-collected successes at an equal data budget. With a coach seeing only camera images, it reaches 56.8%, above both baselines. The action backbone stays frozen throughout.

What it does not show

  • The evidence is the paper's abstract only; experimental setup, task details and full results are not available in the supplied passages.
  • All reported figures, including the 4.6x subgoal gain and the 47.4% to 59.3% improvement, come from the authors and have not been independently confirmed.

Sources

Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 21 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.