MoICL method weights demonstration subsets to cut in-context learning memory strain
Researchers propose treating subsets of demonstrations as experts whose outputs are merged by a learned weighting function, easing memory pressure in in-context learning.
In plain words
Large language models can adapt to a task by reading example demonstrations in their prompt, but long prompts strain memory because transformer processing costs grow quickly with length. The proposed MoICL method instead splits demonstrations into subsets, treats each as a separate expert, and learns how to blend their predictions. The authors report better accuracy and lower inference cost than standard prompting on most datasets tested.
What was announced
In-context learning adapts a model by placing demonstrations in the prompt without changing model parameters, but it treats all demonstrations alike and its transformer cost grows quadratically, which the authors say exhausts memory. MoICL addresses this by assigning subsets of demonstrations to experts and training a weighting function that merges their output distributions using a training set. On their experiments, the approach improved results on 5 of 7 classification datasets against strong baselines, with gains of up to 13 percentage points over ICL and LENS.
The authors also report that MoICL shifts the Pareto frontier of in-context learning by cutting the inference time needed to reach a given performance level with fewer demonstrations. They further claim robustness gains on out-of-domain, imbalanced, and noisy demonstrations, up to 11, 49, and 38 percentage points respectively, and say the method can filter such demonstrations out of datasets. All figures come from the paper's abstract; the excerpt does not cover the experimental setup, model sizes, or which datasets were used.
What it does not show
- Evidence is limited to the paper's abstract; experimental details, datasets, and model configurations are not available for verification.
- Reported gains are the authors' own claims and have not been independently confirmed.
For security and IT teams
- Scope
- The source is an arXiv abstract (v2, replace announcement), so only the authors' summary of MoICL's claims is available, not the full paper.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.