← News

ResearchSources: arXiv · 16 Nov 2026

Balanced accuracy can miss a dropped class in test-time adaptation, arXiv paper argues

A new arXiv study of neural-signal decoding shows aggregate scores can stay flat while a model silently stops predicting one class for some subjects.

In plain words

Test-time adaptation means a model keeps updating itself as new data arrives, and its performance is usually judged by a single average score. The paper shows that this average can look unchanged even when the model stops predicting one class for some patients. The authors say class-level results should be reported alongside the average.

What was announced

The paper, posted to arXiv, examines transductive test-time adaptation for neural-signal classification, where each batch is scored after the model updates on it. The authors evaluated full test-time adaptation across 11 dataset-encoder cells spanning EEG and ECG datasets (Siena, PTB-XL and Sleep-EDF), keeping splits and source-model evaluation fixed, using the subject as the unit, Holm-corrected exact sign tests and bootstrap intervals with a dependence audit.

Their central finding is that balanced accuracy gives a one-class decoder the same value as one reporting both classes, so the aggregate cannot reveal a class that stopped being predicted. In their runs, a class covering about a tenth of a subject's samples went unpredicted in 26-47 of 112 subject-cells, versus 7-9 for controls. Aggregate and within-subject contrast measures named different encoders in 10 of 11 cells. The authors caution the results are bounded by their protocol, including a stream-interleaving effect that reversed sign with state sharing, and recommend reporting class-level and subject-level change alongside aggregate scores.

What it does not show

  • The evidence is the paper's abstract only; full methods, datasets and results could not be checked independently.
  • The reported counts and encoder-pair results are descriptive and conditional on the specific cells, update rules and stream constructions used, so they may not generalise.
  • No independent replication or confirmation of the findings is available.

For security and IT teams

Scope
The findings apply to transductive test-time adaptation in neural-signal classification (EEG and ECG) under the paper's fixed splits, update rules and stream constructions, not to adaptation methods generally.

Sources

Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 17 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.