EasyOPD framework unifies on-policy distillation for language models on verl
A new open framework runs several on-policy distillation methods through one shared backend, addressing fragmented implementations in language-model distillation research.
In plain words
Distillation is a way to train a smaller or newer language model by learning from a stronger teacher model. EasyOPD is a framework that lets the teacher supervise the student while the student generates its own outputs, and it runs several such methods on one shared system. The authors released it with configuration files, documentation, and a demonstration package.
What was announced
The authors of the EasyOPD paper say conventional distillation usually trains a student model on fixed, teacher-generated data, which may not match the states an evolving student actually reaches. On-policy distillation instead gathers teacher or evaluator feedback on outputs the student produced itself. They argue existing methods vary widely in supervision form, tokenizer compatibility, teacher access, and supervision granularity, producing fragmented implementations that are hard to reproduce or extend. EasyOPD is built on verl, a distributed reinforcement-learning framework for large language models, and separates user configuration, method-specific supervision logic, and execution.
The framework connects method modules to a shared backend through extension points for loss construction, rollout metadata, reward processing, tokenizer alignment, and teacher-side computation. The authors instantiated representative methods for three settings: cross-tokenizer distillation, on-policy self-distillation, and step-wise distillation. They report experiments on reasoning, code-generation, scientific-knowledge, and tool-use benchmarks in which the implementations ran on the same verl backend while keeping their method-specific objectives and task-dependent performance profiles. The release includes runnable YAML configurations, documentation, and an installable demonstration package and video.
What it does not show
- The evidence is the paper's abstract only; benchmark results are described at a high level with no numbers, and no independent evaluation is available.
For security and IT teams
- Access
- EasyOPD is released with runnable YAML configurations, documentation, and an installable demonstration package and video, according to the authors.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.