Five-layer audit framework finds UAV intent benchmark accuracy may not reflect radial-velocity understanding
The authors propose a framework that splits benchmark scores into five layers and apply it to the AeroIntent UAV intent benchmark, reporting weak evidence for a radial-velocity readout.
In plain words
A benchmark score is often taken to mean a model has learned the skill the benchmark tests. This paper proposes checking that assumption in five separate layers, from raw data quality to final behaviour, and applies the check to a drone intent-recognition benchmark. The authors report the underlying signal is hard to measure and that accuracy can be achieved without it.
What was announced
Writing on arXiv, the authors propose a five-layer framework that decomposes the assumption that benchmark accuracy shows a model understands the measured construct, separating observation, representation, decoding, use, and behaviour. They instantiate it on AeroIntent, a UAV intent-recognition benchmark. At the observation layer, they report the data is unreliable on 7 of 20 Drone programs, with the direct visual correlate of radial velocity floored at a 0.26% scale change. The representation layer shows two task-designed RGB features with nominal signals: RAFT flow radial bias at r=+0.257 (p=0.030) and bounding-box optical-flow divergence at r=+0.203 (p=0.029).
At the decoding layer, the authors set the minimum detectable effect at |r|=0.599 at the multi-feature Bonferroni threshold, while the largest positive global out-of-sample correlation was r=+0.080, with power 0.00. Injection experiments supported this threshold: aligned injections were detected and orthogonal ones were not. The use layer found accuracy consistent with program memory and static cues, and the behaviour layer found appearance-based accuracy of 0.6552 versus 0.4080 for bounding-box trajectory, a cluster-level paired difference of +0.2471 (p=0.0132), which the authors say is consistent with accuracy not requiring a radial-velocity readout.
What it does not show
- The evidence is the paper's abstract only; the full article's methods and results were not reviewed, so the reported figures cannot be independently checked.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.