LitArc-Bench tests whether AI models track literary reader response beyond position and word cues
A new benchmark separates narrative position, affective vocabulary, and text-specific response dynamics in model predictions of human emotional reactions to literature.
In plain words
When an AI model's emotional predictions follow a human reader's path through a story, that match could come from simple cues like where the passage sits in the narrative or which emotion words it contains. LitArc-Bench is a new benchmark built to separate these explanations. Using Chinese literary excerpts and hundreds of reader questionnaires, the authors show models do capture text-specific response patterns, though with calibration limits.
What was announced
The benchmark consists of 39 Chinese-language literary excerpts, each segmented into seven checkpoints, with 430 questionnaires from 81 adults in two classroom groups yielding 9,020 VAD ratings. The authors compare three future-blind model runs against leave-one-text-out position priors, lexical affect, and human-human references. Position alone reaches r = .470 for arousal, which the authors say makes raw alignment an insufficient measure of model skill.
The results suggest genuine text-specific signal. All six valence and arousal transition advantages over a joint position-lexicon control have text-bootstrap intervals above zero, and cross-fitted models match the correct human text better than 38 mismatched texts in all nine model-dimension cells (p < .0001), while a text-invariant negative control returns chance. On human transitions of at least 0.5 points against the position trend, models recover 69-76% of valence directions and 53-68% for arousal and dominance. The authors note that high shape agreement still coexists with scale bias, and that joint-control advantages are robust for valence but directional and less certain for arousal.
What it does not show
- The evidence is the paper's abstract only; full methods, data availability, and peer-review status are not covered.
- All results are reported by the paper's authors; no independent confirmation is available.
- The benchmark covers Chinese-language literary excerpts and adult classroom participants, so generalization to other languages, texts, or populations is not established by this evidence.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.