← News

ResearchSources: arXiv · 19 Nov 2026

Study links in-context learning and hallucination to one internal mechanism

A new arXiv paper argues that large language models following prompt rules and models drifting from provided evidence can be explained by a single internal mechanism rather than two separate ones.

In plain words

Large language models can pick up rules from examples in a prompt, but the same models sometimes ignore the evidence they are given and produce wrong answers. The paper's authors analyze both behaviors with one framework and find they differ in how strongly an internal signal, tied to each rule's role, shapes the model's output decisions.

What was announced

The authors observe that a model can internally represent role information without that representation reliably constraining what it outputs. Using a first-order decomposition, they characterize the role effect and identify two necessary conditions for it to have substantial influence: sufficient contribution magnitude and coherent signed aggregation. In other words, the role signal must be strong enough and its components must add up coherently for the rule to steer the response.

Causal analysis, according to the paper, shows that role span contributions affect key token choices involving response structure, content selection, and factual values. Role-consistent choices depend on sufficient role span support at these decisions, and choices that fail can be reversed when that support overcomes the deficit. The authors conclude that each functional role defines a pattern of output decisions over which its contributions must remain effective, and differences in this organization explain why in-context learning and out-of-context hallucination appear as opposing outcomes.

What it does not show

  • The evidence is the paper's abstract only; the full article's methods, models, and experimental details are not available for verification.
  • The findings are the authors' own analysis and have not been independently confirmed.
  • The abstract does not name the models tested, quantify results, or describe practical mitigation of hallucination.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.