Frozen language model plus external register tracks long event streams
A new arXiv paper reports that a small trained selector on a frozen model, paired with an integer register, reliably applies ordered changes across very long event sequences.
In plain words
The paper describes a way to make a language model handle long streams of changes without retraining it. The model only lists what changed at each step, and a simple external counter applies those changes in order. This split let even a small model follow tens of thousands of events almost perfectly.
What was announced
The authors train a selector that runs on a frozen language model, so the model itself is not updated. With API-only access, the model lists each transition, and an external integer register applies them in sequence; a confidence rule can flag uncertain updates or keep several candidates. The task studied is composing permutations of five items, which fixed-depth transformers are conjectured not to solve at every input length.
In tests, a 27B model answered 3 of 100 two-event windows without visible reasoning but 96 of 100 when writing intermediate steps. A frozen 0.8B model with the trained selector read every event in 299 of 299 long windows, and a repeat run on a fresh machine passed 298 of 299, just missing a 0.99 confidence-bound target. On 1,000-event documents, all seven models tested both ways succeeded with extraction and the register. Median time dropped from 136.1 to 10.4 seconds for Sonnet 5.5 and from 313.5 to 28.0 seconds for GPT-6.1 Sol, with per-document cost down 55% and 44% respectively. A regex executor also solved all fixed-format documents, while a 4B reader reached 68.8% on free-form comments versus a 61.0% baseline.
What it does not show
- The source is the paper abstract only, so methods, full results and peer review status cannot be verified from the supplied text.
- The near-perfect reliability figures come from the authors' own reported runs and have not been independently confirmed.
- The theoretical claim rests on the stated TC0 vs NC1 conjecture, which remains unproven.
For security and IT teams
- Scope
- Results cover permutation-composition tasks, fixed-format documents and issue-tracker comments described in the abstract; broader production event-stream workloads are not addressed in the supplied evidence.
Sources
Source-support review completed 21 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 21 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.