Raptor tunes speculative decoding trees for vision-language models using rejection feedback
A training-free framework adjusts tree depth, branching width and drafting budget during speculative decoding, reporting up to 2.39x average speedup over autoregressive decoding in the authors' tests.
In plain words
Speculative decoding speeds up image-and-text AI models by drafting several candidate continuations and checking them at once, but long visual inputs make that checking expensive. Raptor is a training-free method that watches which drafts get rejected and uses those signals to resize the drafting tree. The authors report it is faster than standard step-by-step decoding in their experiments.
What was announced
The paper's core observation is that acceptance length alone cannot tell whether drafts fail because too few candidates cover the likely continuations or because the tree simply grows too deep, and the authors argue these two situations call for different resource allocations. Raptor therefore derives rejection-aware signals describing speculative progress, frontier coverage and redundant expansion, and uses them to guide bounded, momentum-smoothed updates to tree depth, branching width and drafting budget.
The same signals also drive execution control that curbs unnecessary drafting and gradually restores expansion when useful. In experiments with two vision-language models across seven multimodal benchmarks, the authors report up to 2.39x average speedup over autoregressive decoding. These figures come from the paper's abstract alone, so independent verification and full experimental details are not available here.
What it does not show
- Evidence is limited to the paper's abstract; the full experimental setup, model versions and benchmark details cannot be assessed from this excerpt.
- The reported speedup is the authors' own claim and has not been independently confirmed.
Sources
Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 21 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.