← News

ResearchSources: arXiv · 19 Nov 2026

CoreQ quantization method adapts error correction per layer in LLM compression

A new post-training quantization framework from an arXiv paper computes a layer-specific correction coefficient in closed form, aiming to reduce overfitting to small calibration sets without manual tuning.

In plain words

Quantization shrinks large language models by converting their stored weights to smaller number formats, so they run on less hardware. The CoreQ paper says a common problem is that errors in early layers change what later layers receive, and existing fixes can overfit the small sample data used for calibration. CoreQ calculates, per layer, how much of that error shift to correct for, using a formula rather than tuning.

What was announced

Post-training quantization maps pretrained model weights to low-bit formats without retraining, usually relying on a small calibration set to minimize a layer-wise objective, the paper's abstract explains. Because layers are quantized sequentially, errors from earlier layers alter the inputs that later layers see, so activations drift away from those of the full-precision model. The authors write that fully applying the empirical mismatch correction can overfit finite calibration data, while a fixed correction coefficient ignores how mismatch estimates vary across layers.

CoreQ, described as learning-free, uses a geometric decomposition of the mismatch to derive a closed-form, layer-adaptive coefficient that controls how much observed mismatch enters each layer's calibration target, then applies greedy successive rounding to a triangular least-squares objective. A bounded beam-search extension, K-CoreQ, trades modest additional compute for improved performance. The authors report lower perplexity and higher downstream accuracy than strong PTQ baselines across multiple LLM families, scales, bit-widths, and quantization settings, with code on GitHub.

What it does not show

  • Evidence is limited to the paper's abstract; no independent testing, benchmark figures, or third-party confirmation of the reported performance gains are available.
  • The abstract does not specify which LLM families, model sizes, or bit-widths were tested, nor the size of the gains over baselines.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.