WeightBench tests whether language models can read, fix and write neural network weights
The benchmark's authors report that frontier agents can already read small networks and write working control policies, but still cannot patch language models or compress them without data.
In plain words
A research team has built a benchmark called WeightBench that measures how well large language models can work directly on the internal parameters of other neural networks, a task they call Weight-Space Programming. The models are tested on reading, repairing and creating network weights without standard training procedures. Results described in the paper show current models handle small networks but struggle with larger ones.
What was announced
The benchmark covers 42 tasks in five tracks: reading weights, fixing them, writing new ones, recognizing impossible edits, and compressing networks without training data. The target networks range from 2K-parameter control policies to 7B-parameter language models. According to the paper, frontier agents can read small networks and produce working control policies without running them, but cannot yet patch language models, compress them without data, or reliably distinguish an impossible edit from a merely hard one.
On the full 42 tasks, the authors report Claude Opus 5.5 scored 0.535 macro, with Sonnet 5.5 at 0.466, Haiku 5.5 at 0.402 and DeepSeek v4-flash at 0.215. GPT-6.1-sol reached 0.568 on the 38 tasks it was run on, against Opus's 0.546 on that subset. The code, task registry, sandbox harness and leaderboard are available on GitHub.
What it does not show
- The evidence is the paper's abstract, which is truncated mid-sentence; the full methodology and results are not available for verification.
- Scores are as reported by the authors and have not been independently confirmed.
For security and IT teams
- Access
- The authors state that the code, task registry, OS-level sandbox harness and live leaderboard are publicly available on GitHub.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.