← News

ResearchSources: arXiv · 20 Nov 2026

Researchers compile exact arithmetic circuits into pretrained language model weights without training

An arXiv paper describes a compiler that grafts deterministic arithmetic circuits into existing language model weights with zero training examples, reporting near-perfect multi-digit accuracy on two models.

In plain words

Large language models often make mistakes on long arithmetic. The paper's authors say their method writes exact calculation steps directly into a model's existing weights, with no training examples and no extra runtime tools. They report high accuracy on six-digit sums, products and differences for two open models, while claiming the models' normal language ability is preserved.

What was announced

The authors, writing on arXiv, frame the work as a response to a known weakness: pretrained language models stay unreliable on exact multi-digit arithmetic unless they are fine-tuned or given external tool interpreters. Their approach uses closed-form, zero-gradient circuit-grafting primitives that compile deterministic computation directly into pretrained weights, using zero training examples and zero runtime hooks. The construction bounds per-layer normalization drift and, the authors state, preserves natural-language competence. Mechanically, it combines a two-layer Mellin-GLU quadrature over bias-free SwiGLU and GeGLU layers, cross-token scale cancellation via signed Chebyshev Indefinite-RoPE self-sink attention with memoryless SiLU-antisymmetric GatedDeltaNet shifts, and axis-aligned width extension with norm compensation, Base-1000 canonical BPE emission and differential tied unembeddings.

An automated five-stage compiler synthesizes circuits for four-to-eight digit signed addition, subtraction and multiplication in constant TC0 depth, alongside Boolean and five-bit Majority/Parity circuits the paper reports at 100% accuracy. On six-digit arithmetic in BF16, the authors report that IBM Granite 4.2 3B reached 99.17% ALU-prefix accuracy and Alibaba Qwen 3.5 4B reached 100.00%, rising to 100% with an FP32 sidecar, and both achieved 98% to 100% mid-sentence resumption. They say this outperforms a 12.8K-sample LoRA comparison at 42.2% and 61.7%. The grafted non-zeros stay small, at most 0.036% and 0.057% of active weights, and a factorized block-sparse sidecar holds A100 memory at 7.89 or 10.65 GiB. All 14 pre-registered long-context Tango splits passed with no out-of-distribution triggers reported.

What it does not show

  • The evidence is the paper's abstract only; the full article's methods, proofs and experimental detail were not available for review.
  • All accuracy, memory and comparison figures are author-reported on two named models and have not been independently confirmed.
  • The abstract does not state whether the compiled models or the compiler are publicly available.

For security and IT teams

Scope
The reported results cover six-digit signed arithmetic on IBM Granite 4.2 3B and Alibaba Qwen 3.5 4B in BF16, with FP32-sidecar variants, not general model reliability.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.