← News

AI securitySources: arXiv · 16 Nov 2026

Paper argues bigger LLMs are not always better at security

An arXiv paper claims security generalization error in LLMs is U-shaped in model size, with an optimal mid-size model rather than ever-larger ones.

In plain words

A research paper argues that the usual rule of thumb for language models — bigger means better — breaks down for security tasks. The authors say large models can silently fix anomalous inputs, such as attack traffic, before security checks see them, hiding problems. Their theory and experiments point to an optimal, mid-sized model for security work.

What was announced

The authors, writing on arXiv, accept that scaling laws hold for general language tasks but argue the corollary fails in security, where decisions hinge on long-tailed anomalous inputs such as zero-day attacks that likelihood fitting serves poorly. They add that the very capability scale brings — inferring an intended whole from a corrupted part — lets a model silently repair an anomaly before any downstream security head sees it.

To formalize this, the paper balances a model's cognitive capability about the world distribution against the security-information retention that survives in its intermediate representation, linked by three information-theoretic bounds: restoration accuracy capped by capability, retention capped by restoration error, and a Fano lower bound on security-task error. From these the authors derive their condensing law: usable security information is unimodal in model size, so security generalization error is U-shaped and minimized at a finite optimal size rather than falling without bound. Experiments on various real-world LLMs are reported to validate the law.

What it does not show

  • Only the paper's abstract is available, so the theorems, experimental setup and results cannot be independently checked from this evidence.

Sources

Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 17 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.