← News

Models & toolsSources: arXiv · 16 Nov 2026

ROCKET: training-free compression allocates layer budgets via knapsack and calibration

Researchers behind ROCKET say the method picks each layer's compression level to limit reconstruction error, then sparsifies weights using a small calibration set, reporting strong results without fine-tuning.

In plain words

ROCKET is a way to shrink large AI models so they run with fewer parameters. Instead of retraining, it uses a small calibration sample to decide how much to compress each layer and to rewrite the weights in one step. The authors report it keeps most of the original model's performance.

What was announced

The method treats compression allocation as a multi-choice knapsack problem, choosing a compression level per layer to minimize total reconstruction error under a target model size. Its second component is a single-step sparse matrix factorization inspired by dictionary learning: weight coefficients are sparsified according to activation-weights sensitivity, and the dictionary is updated in closed form via least squares, avoiding iterative optimization, sparse coding, or backpropagation, according to the paper.

The authors report that ROCKET outperforms factorization, structured-sparsification and dynamic compression baselines across architectures at 20-50% compression rates, retaining over 90% of original performance at 30% compression with no fine-tuning. With light fine-tuning, they say compressing Qwen3-14B to an 8B-parameter model and healing it with 30 million tokens yields performance nearly on par with the original Qwen3-8B. Code is available on GitHub. These are the authors' own claims from the abstract; the excerpt does not include benchmark tables or independent verification.

What it does not show

  • Evidence is limited to the paper's abstract, so benchmark details, experimental setup and comparisons cannot be checked from the supplied passages.
  • All performance figures are the authors' own reports; no independent confirmation is available.

For security and IT teams

Access
The authors point to code at github.com/mts-ai/ROCKET, though the abstract does not describe licensing or release status.

Sources

Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 17 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.