Balanced accuracy can miss a dropped class in test-time adaptation, arXiv paper argues
A new arXiv study of neural-signal decoding shows aggregate scores can stay flat while a model silently stops predicting one class for some subjects.
In plain words
Test-time adaptation means a model keeps updating itself as new data arrives, and its performance is usually judged by a single average score. The paper shows that this average can look unchanged even when the model stops predicting one class for some patients. The authors say class-level results should be reported alongside the average.
What was announced
The paper, posted to arXiv, examines transductive test-time adaptation for neural-signal classification, where each batch is scored after the model updates on it. The authors evaluated full test-time adaptation across 11 dataset-encoder cells spanning EEG and ECG datasets (Siena, PTB-XL and Sleep-EDF), keeping splits and source-model evaluation fixed, using the subject as the unit, Holm-corrected exact sign tests and bootstrap intervals with a dependence audit.
Their central finding is that balanced accuracy gives a one-class decoder the same value as one reporting both classes, so the aggregate cannot reveal a class that stopped being predicted. In their runs, a class covering about a tenth of a subject's samples went unpredicted in 26-47 of 112 subject-cells, versus 7-9 for controls. Aggregate and within-subject contrast measures named different encoders in 10 of 11 cells. The authors caution the results are bounded by their protocol, including a stream-interleaving effect that reversed sign with state sharing, and recommend reporting class-level and subject-level change alongside aggregate scores.
What it does not show
- The evidence is the paper's abstract only; full methods, datasets and results could not be checked independently.
- The reported counts and encoder-pair results are descriptive and conditional on the specific cells, update rules and stream constructions used, so they may not generalise.
- No independent replication or confirmation of the findings is available.
For security and IT teams
- Scope
- The findings apply to transductive test-time adaptation in neural-signal classification (EEG and ECG) under the paper's fixed splits, update rules and stream constructions, not to adaptation methods generally.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.