Edit-based training targets first failure points to shape LLM reasoning
Researchers propose supervising models only on minimal corrections to their own reasoning traces, reporting better instruction compliance than full-trajectory imitation or trajectory-level reinforcement learning.
In plain words
When AI models are trained to reason, they are usually rewarded or penalized for whole answers, so they never learn exactly where their thinking went wrong. The proposed method finds the first point where a model's reasoning fails and trains it only on that small corrected span. In a test with the Qwen3-32B model, this improved how well the model followed reasoning instructions.
What was announced
The authors argue that existing training signals are too coarse: supervised fine-tuning on full trajectories forces a model to imitate an entire demonstration, while trajectory-level reinforcement learning rewards or penalizes whole completions without identifying where they succeed or fail. Edit-based training instead has an editor locate the first failure under a chosen criterion and propose a minimal replacement span, with supervision applied only to those edited spans. This concentrates learning where reasoning first breaks down while preserving existing patterns elsewhere.
The framework can run offline or online, with the online variant adapting supervision to the model's evolving failure modes. Applied to reasoning instruction following with Qwen3-32B on the ReasonIF benchmark, the authors report substantially better compliance with reasoning instructions than full-trajectory imitation and trajectory-level reinforcement learning baselines, while largely preserving skills on untargeted tasks. They position localized correction as a complement to existing approaches rather than a replacement.
What it does not show
- Evidence is limited to the paper's abstract; detailed methods, benchmark numbers and peer review status are not available in the supplied excerpt.
- Results come from a single model (Qwen3-32B) and a single benchmark (ReasonIF), so generalization to other models or tasks is not established by this evidence.
- The reported improvements are the authors' own claims and have not been independently confirmed.
Sources
Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 21 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.