← News

Models & toolsSources: arXiv · 20 Nov 2026

Post-training changes how language models read out what they represent, study finds

A new arXiv study comparing six open-weight base models with their post-trained versions concludes that fine-tuning altered how internal knowledge was turned into answers, not what the models represented.

In plain words

Language models can hold information internally that they fail to use correctly when answering. Researchers found that post-training, the extra tuning after initial training, changed how that stored information was read out into answers. A small amount of correctly labeled training data was enough to shift the direction of this effect.

What was announced

The authors compared linear probes, which decode information from a model's internal states, with the models' own answers, analyzed using signal detection theory. Across six open-weight base models and their post-trained versions, probes and word n-gram classifiers separated human-written from AI-written German texts well, while the answers did not and often ranked the texts inversely. Post-training lowered answer sensitivity on average, with a reported change in d' of -0.34 across model families and -0.29 across three prompts. It also changed how a stated base rate was used: base models shifted their log-odds by less than the stated change in prior odds, while most post-trained models shifted by more.

The picture reversed on a second task, judging the truth of negated statements, which the authors added as a reproducible case because their authorship corpora cannot be shared. There, probes decoded truth at or near ceiling in every model, but several base models failed to use it in their answers, and post-training repaired this. In a training study, adding 2,000 correctly labeled authorship judgments to otherwise identical fine-tuning data improved answers on newspaper and textbook corpora while inverting them on chats, where answers followed a news-trained probe of the base model. The authors conclude that post-training changed how a judgment is read out of what a model represents, and that a small training signal can set its direction.

What it does not show

  • The evidence is the paper's abstract, so methods, caveats and full results are not available for verification.
  • The authorship corpora cannot be shared, so the main results cannot be independently reproduced; only the negated-statements case is reproducible from public data.
  • The study covers six open-weight model families and two examined tasks, and the abstract does not establish how findings generalize to other models or tasks.

For security and IT teams

Scope
Findings apply to six open-weight base models and their post-trained versions on German authorship judgments and negated-statement truth tasks, per the abstract.

Sources

Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 21 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.