PhysAlign benchmark shows multimodal models misassign physics diagram roles despite accurate recognition
A new arXiv benchmark separates visual recognition from physical-role assignment in physics diagrams, finding that even correctly recognized values are often linked to the wrong entity or scope.
In plain words
Multimodal AI models can read values and symbols in physics diagrams, but they must also attach each element to the right physical entity or condition. PhysAlign is a benchmark testing whether models make that connection correctly. The authors report that models often recognize the content yet still assign it to the wrong role.
What was announced
PhysAlign contains 3,341 human-validated probes drawn from 986 physics problems. Using localized probes and controlled variants, the benchmark separates correspondence errors from recognition failures, and the authors introduce five evaluation metrics, including CAcc, GAcc, and JAcc, covering recognition, correspondence, and problem solving.
Across the evaluated models, the paper reports a consistent gap between recognition and grounding. When the queried content was correctly recognized, the conditional correspondence error rate was 13.8% for GPT-6-Astra and about 50.6% for InternVL3.5-8B. The authors argue that strong perception alone does not ensure reliable physical interpretation, and that answer-level accuracy can hide this grounding bottleneck.
What it does not show
- The evidence is the paper's abstract only; the full methodology, model list, and experimental details are not available for verification.
- The reported error rates come from the authors' own evaluation and have not been independently confirmed.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.