XMerge trims transformer layers from LLMs without changing the serving architecture
A post-training method removes whole transformer layers and re-fits a neighbouring block, reporting stronger quality than five baselines across seven Llama and Qwen models.
In plain words
Large language models can run faster when some internal layers are removed, but existing removal methods often damage output quality unpredictably. XMerge is a post-training technique that picks which layer block to drop and then adjusts a surviving neighbouring block so the model's behaviour stays close to the original. It needs no task labels, no fine-tuning, and adds nothing at inference time.
What was announced
The arXiv abstract describes two components. Cross-axis selection finds a block whose hidden states show low relative-magnitude and angular change, and local boundary reconstruction re-fits the adjacent surviving block to match the original two-block output. The authors say this avoids task labels, end-to-end fine-tuning, architectural changes, and extra inference-time parameters.
Across seven Llama and Qwen backbones from 0.5B to 8B parameters, five published baselines, and three layer-reduction levels, the reported advantage is largest at the most aggressive removal. At k=4 the paper claims first place on six of seven backbones on CORE, a 22-task aggregate, and on six of seven on MMLU, while avoiding the large perplexity increases of several competing operators. A task-level bootstrap found the 95% confidence intervals for the three largest CORE margins exclude zero, with remaining margins consistent with ties. The authors also report it never collapses across the 14 model-regime cells, ranking top-2 in both zero-shot and in-context settings, and ablations attribute most of the gain to local reconstruction. The construction cost is said to be recovered through per-token decode savings after roughly tens of thousands of requests.
What it does not show
- The evidence is the paper's abstract only; the full article, methodology and reproducibility have not been reviewed.
- All quality, calibration and cost-recovery figures are the authors' own claims and have not been independently confirmed.
- Calibration results come from a single-backbone probe, and several reported margins are consistent with ties per the authors' own bootstrap.
For security and IT teams
- Scope
- XMerge introduces no architectural changes or additional inference-time parameters, so serving infrastructure would remain standard if the method is adopted.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.