New EscherVerse benchmark finds vision-language models far behind humans on goal-driven spatial reasoning
A research team's Teleo-Spatial Intelligence benchmark tests whether AI models can connect physical dynamics with human intentions, and 27 leading models still trail human annotators by a wide margin.
In plain words
Researchers built a large test called EscherVerse to check whether AI systems that process images and text can reason about how objects move and why people act. The best model answered about 57% of questions correctly, while people scored between roughly 85% and 95%. Fine-tuning on the team's data helped open-weight models but did not close the gap.
What was announced
The authors introduce Teleo-Spatial Intelligence (TSI), a form of spatial reasoning that combines physical dynamics with action goals to predict what an action will do to a scene. They argue that current vision-language models, despite strong results on controlled benchmarks, often fail to jointly model physical dynamics, reference frames, and the latent human intentions behind spatial change in open-world environments.
To measure this, the team built EscherVerse from 11,328 real-world videos, comprising an 8,000-example benchmark and a 35,963-example instruction-tuning set. Evaluating 27 state-of-the-art vision-language models alongside first-pass responses from 11 human annotators, they report that the strongest proprietary model reached 57.26% overall accuracy, well below the human range of 84.81% to 95.14% (mean 90.62%). Fine-tuning open-weight models on the intent-aware data narrowed the gap without eliminating it. The authors position EscherVerse as a diagnostic testbed for purpose-aware spatial reasoning rather than evidence that any model has solved it.
What it does not show
- The evidence is the paper's abstract only; full methodology, task design and error analysis are not available in the supplied passages.
- Human performance figures come from first-pass responses by 11 annotators, a small sample described by the authors themselves, not an independent evaluation.
- The accuracy figures are the authors' own measurements and have not been independently confirmed.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.