EDA adapts speculative decoding draft models to fine-tuned LLMs without full retraining
Researchers propose Efficient Draft Adaptation, a framework that realigns draft models with fine-tuned target models using shared and private components plus regenerated training data, cutting retraining costs.
In plain words
Speculative decoding speeds up large language model inference by using a smaller draft model to predict tokens that a larger target model checks. When the target model is fine-tuned for a specific domain, the draft model no longer matches it well and the speedup drops. EDA is a method from the paper's authors that re-adapts the draft model cheaply by updating only a small private component and reusing data generated by the fine-tuned target model.
What was announced
The authors of the arXiv paper say speculative decoding degrades when target models are fine-tuned for specific domains, and that retraining a draft model for every target is costly. EDA addresses this with three parts: a decoupled architecture separating shared and private components so only the lightweight private part is updated; a data regeneration strategy in which the fine-tuned target model regenerates training data to better match speculative decoding; and a sample selection mechanism that prioritizes high-value data. In the authors' experiments, EDA restored speculative performance on fine-tuned models, achieving higher average acceptance lengths at significantly reduced training cost compared with full retraining, and code was released on GitHub.
What it does not show
- Evidence is limited to the paper's abstract, so experimental details, model sizes and baselines cannot be verified from the supplied passages.
- Performance claims are the authors' own and have not been independently confirmed.
For security and IT teams
- Status
- The authors state code is available at the GitHub repository Lyn-Lucy/Efficient-Draft-Adaptation.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.