New tumbling-E benchmark measures temporal reliability for human-AI comparison

A new preprint from researchers Avneet Sandhu and Bin Hu introduces a dynamic computerized tumbling-E test designed to measure not just static visual acuity but the temporal reliability of human sequential perceptual decisions. Unlike traditional chart-based tests that yield a single acuity score, this adaptive staircase task records reaction times, timeout rates, and trial-level adaptation patterns. Over 1,154 valid trials from 21 participants across 77 sessions, the study captured a mean reaction time of 1,546 milliseconds, a timeout rate of 6.6%, and a smooth adaptation from 29.42 arcminutes at the start to 5.04 arcminutes by trial 19—converging near 20/20 level. These results demonstrate fast, reliable human timing and provide a richly annotated dataset of how perception unfolds over multiple decisions.

The key contribution is the introduction of the Temporal Hallucination Index (THI), which quantifies how static accuracy can mask delays, drift, persistence, and unstable convergence in decision-making. By converting the classic tumbling-E task into a temporally resolved benchmark, the authors create a human-only baseline for future comparison with artificial agents. This matters for AI safety and alignment: as AI systems increasingly make sequential perceptual decisions (e.g., in autonomous driving or medical imaging), understanding human timing patterns helps identify where models may appear accurate but deviate in temporal reliability. The dataset is publicly available for researchers working on human-AI comparison in perception tasks.