← News

Models & toolsSources: arXiv · 16 Nov 2026

EDA adapts speculative decoding draft models to fine-tuned LLMs without full retraining

Researchers propose Efficient Draft Adaptation, a framework that realigns draft models with fine-tuned target models using shared and private components plus regenerated training data, cutting retraining costs.

In plain words

Speculative decoding speeds up large language model inference by using a smaller draft model to predict tokens that a larger target model checks. When the target model is fine-tuned for a specific domain, the draft model no longer matches it well and the speedup drops. EDA is a method from the paper's authors that re-adapts the draft model cheaply by updating only a small private component and reusing data generated by the fine-tuned target model.

What was announced

The authors of the arXiv paper say speculative decoding degrades when target models are fine-tuned for specific domains, and that retraining a draft model for every target is costly. EDA addresses this with three parts: a decoupled architecture separating shared and private components so only the lightweight private part is updated; a data regeneration strategy in which the fine-tuned target model regenerates training data to better match speculative decoding; and a sample selection mechanism that prioritizes high-value data. In the authors' experiments, EDA restored speculative performance on fine-tuned models, achieving higher average acceptance lengths at significantly reduced training cost compared with full retraining, and code was released on GitHub.

What it does not show

  • Evidence is limited to the paper's abstract, so experimental details, model sizes and baselines cannot be verified from the supplied passages.
  • Performance claims are the authors' own and have not been independently confirmed.

For security and IT teams

Status
The authors state code is available at the GitHub repository Lyn-Lucy/Efficient-Draft-Adaptation.

Sources

Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 17 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.