← News

Models & toolsSources: arXiv · 18 Nov 2026

Stress test finds multimodal AI grading assistant awards full credit to distorted graphs

An exploratory arXiv study stress-tested a multimodal large language model grading assistant on kinematics graph reading and found some grades lacked graphical support.

In plain words

Researchers tested whether an AI model that grades by reading images could reliably score physics graphs about motion. The model sometimes judged correctly, but it also gave a badly distorted graph full marks while cutting credit for other answers. The authors say teachers should review its scores, including perfect ones, before using them as final grades.

What was announced

The study, published as an arXiv abstract, examined selected grading outputs for synthetic responses to four kinematics exercises, using visual distortion and progressive transformation toward reference graphs across different model and prompt configurations. In some cases the judgments were sound: the model distinguished instantaneous zero velocity from rest over a finite interval and recognized positive curvature. But a severely distorted velocity–time graph received full credit despite disconnected and branching traces and an unclear zero-velocity reference, while other submissions graded under the same configuration lost substantial credit.

The authors also note that numerical cutoffs meant to separate meaningful features from drawing irregularities were supplied without a coordinate convention for unnumbered graphs, leaving the basis of exact partial-credit awards unresolved. They argue that stress testing can expose unsupported awards before classroom use, and conclude that, for the configurations examined, scores should not be accepted as final grades without instructor review, including review of full-credit responses.

What it does not show

  • The evidence is an abstract describing an exploratory study of selected outputs on four exercises, so it does not establish how the models perform beyond these configurations and cases.
  • The source does not resolve how exact partial-credit awards were determined, since no coordinate convention was supplied for unnumbered graphs.

For security and IT teams

Scope
Findings apply only to the model and prompt configurations examined on four kinematics exercises, and the authors do not recommend accepting scores as final grades without instructor review.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.