DACE paper proposes co-evolution training to keep LLM safety defenses ahead of shifting attacks
A new arXiv paper describes a training framework in which attacker and defender models improve together, aiming to fix two failure modes the authors identify in current adversarial safety alignment.
In plain words
Researchers behind a new preprint say AI safety training can fall behind when attack methods keep changing. Their proposed method, DACE, trains an attack generator and a defensive model side by side, pushing the attacker to try varied strategies and the defender to keep practicing older threats. The authors report stronger robustness on their benchmarks without hurting general performance.
What was announced
The paper argues that current co-evolutionary alignment pipelines, where a defensive model is trained against an evolving attacker, suffer from two linked problems. The attacker can settle on a narrow set of rewrites that score well, which the authors call attack strategy collapse, while the defender can lose the ability to handle earlier attack types as new ones appear, described as defense adversarial forgetting.
DACE addresses both sides at once. The attacker works within a two-dimensional strategy space combining risk category and attack style, rewarded for covering new strategies rather than only producing novel text. The defender draws training examples from a Bayesian replay pool whose threat estimates are updated with time decay and sampled through Thompson sampling, so past attacks remain in rotation. On the benchmarks the authors used, they report better robustness to out-of-distribution attacks, broader attacker strategy coverage, and preserved general capability. The claims come from the paper's abstract alone and have not been independently verified.
What it does not show
- Evidence is limited to the paper's abstract; no full-text results, benchmark numbers, or third-party evaluation are available.
- The reported robustness gains are the authors' own claims and do not constitute independent confirmation.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.