Interactive imitation learning eases what a model must represent, arXiv study finds
A new arXiv paper argues that querying the expert along the learner's own trajectories shifts the burden from copying the expert's policy to matching its value function.
In plain words
Imitation learning trains software to copy an expert by showing it examples. The paper reports that letting the learner act and ask the expert for guidance means it only needs to capture how good the expert's choices are, not copy every choice exactly. This matters when the learner's model is too small to fully mimic the expert.
What was announced
The authors, writing on arXiv, study imitation learning, where an agent is trained to replicate expert behavior from demonstrations, with uses from robotics to language model training. Standard behavior cloning, they note, suffers from compounding errors and plateaus when the learner cannot perfectly represent the expert's policy, a common situation in distillation. Their main theoretical result is that interactive expert queries relax the representational demands: the learner only needs a model able to realize the expert's value function rather than the expert's policy itself. They introduce OVI, an interactive on-policy algorithm they describe as statistically and computationally efficient under that value-function assumption.
The paper also reports a negative result: without stronger assumptions, any offline imitation learning algorithm must scale with the complexity of the expert policy class, which the authors present as evidence that interaction is necessary. Empirically, they say OVI outperformed offline policy-based behavior cloning, interactive policy-based DAgger, and offline value-based methods, with the largest gains when the learner network was much less expressive than the expert's. These claims come from the paper's abstract and have not been independently verified.
What it does not show
- The evidence is the paper's abstract only; full proofs, experimental setup and details are not available in the supplied passages.
- Performance comparisons and efficiency claims are the authors' own and have not been independently confirmed.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.