ZeroVLN navigates without fine-tuning the vision-language brain
A research team reports a navigation framework that leaves the vision-language model untouched during supervised training, relying on reinforcement learning and a language interface instead.
In plain words
Vision-language models can guide robots, but tuning them for navigation often erodes their broader skills. ZeroVLN instead lets the model issue plain-language movement commands to a separate action module, then trains both parts with reinforcement learning. The authors say this keeps the model's general abilities largely intact.
What was announced
The arXiv abstract describes how typical navigation systems attach a trainable action head to a vision-language model and tune the model itself on navigation data so it can predict geometric targets. That adaptation, the authors note, can shift the base weights noticeably and weaken the model's wider visual competence. ZeroVLN takes a different route: the model communicates movement choices through a semantic tool-calling interface, phrased in natural language, to a dedicated action expert that carries out the requests.
Training proceeds in two stages. The action expert is first adapted to execute the movement requests, and closed-loop reinforcement learning then refines both the model's decisions and the expert's continuous control on the same navigation episodes. According to the paper, the resulting system beats representative supervised hierarchical baselines on the standard VLN-CE benchmarks while changing the base model's weights only slightly. The authors position it as the first slow-fast navigation framework of this kind, though the claims rest on their own reported results and the abstract alone.
What it does not show
- Evidence is limited to the paper's abstract; no independent evaluation, full methodology, or detailed benchmark numbers are available in the supplied passages.
Sources
Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 21 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.