← News

ResearchSources: arXiv · 20 Nov 2026

ICARE framework targets catastrophic forgetting in multi-task NLI training

Researchers propose ICARE, a framework pairing selective example choice with knowledge distillation to help natural language inference systems learn new tasks without losing old ones.

In plain words

When a language system is trained on one task after another, it can lose its grip on earlier tasks, a problem called catastrophic forgetting. ICARE addresses this by carefully choosing which training examples the system sees and by transferring knowledge between tasks. The authors report better accuracy on new tasks and less forgetting of old ones.

What was announced

The paper's abstract frames the problem around fine-tuned large language models that depend on natural language inference systems trained across varied tasks such as sentiment analysis, news classification and coreference resolution. Continual multi-task training can cause catastrophic forgetting, where previously learned tasks are lost, particularly in domains without specific labels. The authors add that rehearsing past labelled examples can worsen overfitting and hinder classification of new categories.

ICARE combines two components. An instance selection module, built on representation learning, chooses which training examples are shown to the NLI system. A knowledge distillation method then transfers both feature-level and decision-making (logit-level) knowledge between tasks, using pruning techniques and optimised objective functions. According to the abstract, evaluations on benchmark text classification datasets showed a significant improvement in target accuracy and a reduction in the forgetting metric, and the source code is available on GitHub.

What it does not show

  • Evidence is limited to the paper's abstract; no independent evaluation, specific benchmark scores or comparisons with named baselines are provided in the supplied passages.

For security and IT teams

Access
The authors state that ICARE's source code is publicly available in a GitHub repository.

Sources

Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 20 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.