← News

Models & toolsSources: arXiv · 18 Nov 2026

Study ties chat model safety shifts after 4-bit quantization to specific rounding error

A pre-registered study of three small chat models finds that 4-bit quantization can change refusal behaviour in opposite directions, and that the specific rounding error, not bit width, drives the change.

In plain words

Quantization shrinks a chat model so it can run on phones and laptops, but the shrinking process introduces small rounding errors in the model's weights. The study found that these specific errors, not the 4-bit format itself, changed how often the models refused harmful requests. That means each compressed build must be checked against its original separately.

What was announced

The authors, publishing on arXiv, ran a pre-registered study of three 3-4B parameter chat models using paired tests, blind human labels and an 8-bit null condition. They report two 4-bit shifts of opposite sign: when a compliant reply was prefilled, Qwen2.5-3B continued into harmful content more often, while on held-out jailbreaks Llama-3.2-3B complied 10.3 points less often. Because a 4-bit build is one of many weight perturbations of similar size, the study tested whether behaviour changes come from the specific rounding error or from any perturbation that large.

Matched perturbation tests separated the two explanations. Shuffled and Gaussian versions of the rounding error averaged close to zero effect, while scaling the real error, from its mirror image to twice its size, changed behaviour near-linearly. At half the real error, Llama's likelihood was almost unchanged, yet it already complied 5.8 points less often. The authors also report that llama.cpp's Q4_K_M format, at the same bits per weight, moved neither model. They conclude that refusal under 4-bit quantization depends on the specific rounding error rather than on bits per weight or likelihood, so each build should be compared with its original.

What it does not show

  • The evidence is the paper's abstract only; full methods, model coverage and statistical detail are not available in the supplied passage.
  • Findings cover three 3-4B chat models and may not extend to other model sizes or families.
  • The reported shifts are the authors' own measurements and have not been independently confirmed.

For security and IT teams

Scope
The study examined three 3-4B chat models (Qwen2.5-3B and Llama-3.2-3B are named) and the llama.cpp Q4_K_M quantization format, so conclusions are scoped to those builds.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.