← News

Models & toolsSources: arXiv · 18 Nov 2026

Lightweight RL selector refines trajectory choice on top of frozen world model

A new arXiv paper describes a residual-guided reinforcement learning framework that fine-tunes a small policy to pick driving trajectories, keeping the world model frozen.

In plain words

Self-driving planners generate many possible paths and must choose one. The paper's method trains a tiny neural network, using reinforcement learning, to rank those candidate paths while a larger prediction model stays unchanged. The authors report modest accuracy gains in open-loop tests on the nuScenes dataset.

What was announced

In automated driving, world models predict how a scene will unfold so candidate trajectories can be evaluated, but fixed selection rules may miss trade-offs that depend on the scene, the paper's abstract states. The proposed framework uses proximal policy optimization to fine-tune a joint policy over candidate selection and bounded trajectory residuals, while a context-conditioned ranking network learns preferences from feedback on local perturbations of selected trajectories. At inference, the selector blends learned preference and predicted risk ranks with fixed weights and returns the chosen candidate unmodified.

The training updates only 16,106 parameters across actor and critic, and the deployed selector keeps just the 11,725-parameter ranking network, with the residual branch and critic used only during training. On open-loop evaluation over the nuScenes validation set, the authors report a 3.99% reduction in average L2 trajectory error versus the RiskWorld baseline without RL-based selection. Under matched budgets for feedback accesses and actor updates, residual exploration cut L2 by 0.83% versus a categorical PPO variant without residuals, and the method stayed below RiskWorld's L2 when risk features were removed or shuffled.

What it does not show

  • Evidence is limited to the paper's abstract; full experimental details, baselines beyond those named, and peer review status are not available.
  • Reported gains are open-loop results on the nuScenes validation set and do not establish closed-loop or on-road performance.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.