Study quantifies how weight edits weaken LLM watermark evidence
An arXiv paper derives exact bounds on how much detection evidence survives when open-weight models are quantized, pruned or fine-tuned after release.
In plain words
Watermarks are hidden statistical signals that let a verifier check whether text came from a specific AI model. The paper examines what happens to that check when someone modifies a released model's weights, and provides mathematical bounds plus test results showing which edits keep the watermark detectable.
What was announced
The authors model any weight modification as a residual in next-token logits at each shared prefix and derive an exact finite-vocabulary lower bound on the expected log-likelihood-ratio evidence under a residual-span constraint, plus a span-only deviation bound and a first-order characterization of evidence shift via a covariance term with an explicit second-order remainder. They also determine the residual span needed to reach a target evidence level. For sequence-level detection, they combine a conditional exponential-moment bound with a supermartingale argument and a distribution-free order-statistic tolerance bound, yielding a high-confidence upper bound on miss probability after calibration and evaluation on independent continuations.
In experiments with OpenStamp on Llama-2-7B and Mistral-7B-v0.3, INT8 quantization and 30% pruning retained test true-positive rates of 85.6-94.8%, while 500-step LoRA fine-tuning lowered them to 28.8% and 11.6%. The covariance prediction tracked the measured mean LoRA evidence drift on both models, with no remainder-bound violations across 160,000 test prefixes, and zero false positives appeared among 500 base-model continuations per model. The authors state these results improve understanding of watermark robustness to model-editing and tuning attacks.
What it does not show
- The source is only the paper's abstract, so methods, proofs and full experimental details could not be checked.
- Results are reported for OpenStamp on two 7B models; robustness on other watermarks or model sizes is not established.
- True-positive retention after LoRA fine-tuning drops sharply, so the bounds do not imply watermark survival under all edits.
For security and IT teams
- Scope
- The reported evidence covers OpenStamp-style verification on Llama-2-7B and Mistral-7B-v0.3 under INT8 quantization, 30% pruning and 500-step LoRA fine-tuning only.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.