← News

ResearchSources: arXiv · 19 Nov 2026

PLANT pretrains label attention for extreme multi-label text classification

Researchers propose PLANT, a plug-and-play way to initialize label attention in multi-label classification models, reporting gains on ICD coding, legal topic classification and content recommendation.

In plain words

Some AI systems must tag a document with many labels at once, such as medical codes. PLANT gives the model a strong starting point for deciding which input words matter, instead of learning this from scratch. The authors say this helps most when labels are rare.

What was announced

The method targets extreme multi-label text classification, where models use multi-label attention to focus on key tokens in the input. Learning good attention weights is difficult, so PLANT initializes the label-attention module that sits on top of an LLM's output representations. It plants label-specific attention using a Learning-to-Rank model trained on mutual information gain, after which the whole model is fine-tuned end to end. The approach is described as architecture-agnostic and was integrated with LLM backbones including Mistral-7B, Llama-3.1-8B, DeepSeek-V3 and Phi-3-mini.

According to the paper's abstract, PLANT outperforms state-of-the-art methods on tasks including ICD coding, legal topic classification and content recommendation, and improves fine-tuned LLMs in controlled comparisons outside the medical domain. The largest gains appear on rare labels, though a training-free LLM extraction baseline remains competitive there. Ablation studies point to attention initialization as a key driver of the improvements. Code is available on GitHub.

What it does not show

  • The evidence is the paper's abstract only; reported results are author claims and have not been independently verified.
  • The abstract does not include benchmark numbers, datasets or comparison details, so the size of the gains cannot be assessed.

For security and IT teams

Scope
The paper covers extreme multi-label text classification tasks such as ICD coding, legal topic classification and content recommendation, not general-purpose LLM use.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.