← News

Models & toolsSources: arXiv · 20 Nov 2026

Study finds LLM-guided weight edits can repair language models without retraining

A controlled arXiv study reports that an editor model choosing projection recipes by hand matched or beat automated search on some faults, with clear limits.

In plain words

Researchers tested whether a large language model, acting as an editor, could fix faults in other language models by making direct numerical changes to their weights, with no gradient training. In their experiments this interactive editing sometimes matched or beat automated search, but not always.

What was announced

The arXiv paper reports a controlled study of Transformer repair in which an editor inspects predictions, activation summaries and weight statistics, then selects projection recipes to apply to a target model without gradient updates. Across 20 recorded sessions covering five masked and causal language models, the authors compared these live selections against fixed projections, scripted and matched-budget black-box search, and low-rank adaptation. Under the core two-layer fault, live selection matched black-box search on SmolLM2 and Qwen2.5, while on Pythia it reached 65.62% completion accuracy against 0% for black-box search and 66.15% for a fixed projection.

Under a separate compound fault affecting attention and feed-forward weights, both scripted and LLM-guided selection raised the descriptive five-checkpoint mean from 24.89% to 71.25% using the same operator family. On unmodified checkpoints, scripted projection produced small gains or losses on a separate set of framed questions. The authors state the comparisons establish the scope of controller-guided repair for the tested faults and completion tasks, noting cases where feedback changed the edit decision, where operator coverage explained recovery, and where supervised adaptation performed better.

What it does not show

  • The evidence is an abstract only, so methods, full results and caveats in the paper could not be checked.
  • The five-checkpoint mean is described by the authors as descriptive, and the tested faults and completion tasks may not generalize to other settings.
  • The abstract reports cases where supervised adaptation performed better, so interactive editing is not uniformly the strongest approach.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.