Researchers steer LLMs to direct answers using hidden-state directions
An arXiv paper reports that a single adjustment to a model's internal state lets direct prompting beat chain-of-thought accuracy on 25 tasks while using far fewer tokens.
In plain words
Large language models often answer questions better after writing out reasoning steps, a technique called chain-of-thought. The paper's authors found a direction inside the model's internal representations that, when the model's state is nudged toward it before answering, lets the model answer directly with higher accuracy and far fewer generated tokens.
What was announced
The authors label each chain-of-thought hidden state as recovering if patching it into a direct prompt yields the correct answer, and failed otherwise. A linear probe showed the model encodes these two classes differently, and the representation space of the direct prompt contains a direction for each question that leads the model to the correct answer in a short generation.
Concretely, the method rotates the direct prompt's last hidden state once, before generation, toward a direction built from that question's own recovering and failed states. Across 25 tasks, this raised the direct prompt's mean accuracy from 46% to 85% on LLaMA 3.1 8B, where chain-of-thought reached 69%, and from 52% to 86% on Qwen 2.5 7B, where chain-of-thought reached 77%, while using about a tenth of chain-of-thought's tokens. The authors conclude that chain-of-thought is not the shortest route to this accuracy and suggest the directions could train models to align their states with them in future work.
What it does not show
- The evidence is an abstract only, so the full paper's methods, task details and evaluation setup are not covered here.
- The reported accuracies come from the authors' own experiments on two models and have not been independently confirmed.
- The direction is built from each question's own recovering and failed states, so the description does not establish how it would work without those labels.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.