CurvDelta builds capability preservation into LLM adaptation updates
The arXiv paper introduces a framework that shapes parameter updates so adapting a model toward a target task leaves its general capabilities nearly unchanged, with a low-rank variant called CurvLoRA.
In plain words
When a large language model is tuned for a new task, it can lose skills it already had. CurvDelta is a proposed method that constrains the tuning update itself, using an approximation of the model's local geometry so the update barely disturbs existing behavior. A lighter-weight version, CurvLoRA, applies the same idea with fewer parameters.
What was announced
The authors frame capability preservation as a central challenge in adapting large language models: an adaptation can hit its target while degrading general abilities. CurvDelta addresses this by building the preservation constraint into the parameterization of the update, analyzing the local geometry around the pre-adaptation model to construct a low-curvature space in which updates leave the preservation loss nearly invariant. The paper says this is made practical for LLMs through Kronecker-factored approximate curvature (K-FAC) and a factorized projection.
The reported evaluation covers mathematical reasoning, code generation, and knowledge unlearning. On reasoning and code generation, the authors say their methods reach competitive downstream performance while better preserving general capabilities than baselines, describing a favorable acquisition-retention trade-off. On unlearning, they report results on the TOFU and MUSE benchmarks showing a better balance between targeted forgetting and retention, with retention close to a from-scratch retraining reference. These are the authors' own reported comparisons; the abstract does not describe independent verification, and the excerpt does not include model sizes, training details, or full results.
What it does not show
- Evidence is limited to the paper's abstract; no full results, model details, or independent confirmation are available.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.