TOMAS links hand pose and object semantics to forecast egocentric hand motion
Researchers present TOMAS, a framework that models how manipulation intentions shape hand trajectories, reporting state-of-the-art results on public benchmarks for hand motion forecasting.
In plain words
A research team introduced TOMAS, a method that predicts how a person's hands will move next in first-person video. It connects the structure of the hand with what object is being handled, then breaks the future motion into ordered stages with learnable timing to guide its predictions.
What was announced
The authors argue that existing forecasting methods do not sufficiently capture how articulated hand movements relate to object semantics, even though different intentions can produce very different trajectories with the same object. TOMAS addresses this with a topology-aware hand-object reasoning mechanism that ties hand skeletal structure and articulation dynamics to object semantics and spatial relationships, jointly inferring likely interaction targets and latent manipulation operations. These object-operation representations then serve as conditions for predicting future hand trajectories, with supervision adapted to whatever annotations each dataset provides.
A second component, an adaptive staging mechanism, organizes future hand movements into ordered latent stages whose temporal boundaries are learned rather than fixed. Stage-specific geometric subgoals guide the refinement of predicted trajectories according to the underlying manipulation pattern. On multiple publicly available benchmarks, the authors report that TOMAS achieves state-of-the-art performance in multi-dimensional and multi-target hand motion forecasting. The evidence for these claims is the paper's abstract; the reported results have not been independently verified, and the abstract does not name the benchmarks or quantify the improvements.
What it does not show
- The source is the paper's abstract only; benchmark names, datasets and quantitative results are not specified, and the reported state-of-the-art performance is not independently confirmed.
Sources
Source-support review completed 20 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 20 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.