VLMs mimic human visual search patterns—but with surprising twists

Farahnaz Wick's new arXiv preprint puts vision-language models (VLMs) through the paces of classic visual-search experiments—the same paradigms that have illuminated human attention for decades. Instead of reaction time, Wick uses the number of reasoning tokens a model spends per trial as an analog of search effort. She adapted four tasks: feature versus conjunction search (pop-out vs. serial), spatial-configuration search (T vs. L), enumeration, and the tilted/vertical search asymmetry. Both frontier models (e.g., those with extensive reasoning tokens) and mid-tier models were tested against a large public human benchmark (Wolfe et al., 2010). The results show several human signatures are reproduced: feature search incurs flat effort, while conjunction search effort climbs with set size; frontier models maintain accuracy where mid-tier models collapse to chance; and a resolution control confirms the conjunction cost is genuine search rather than difficulty resolving small shapes.

But the divergences are equally telling. In humans, target-absent slopes are steeper than target-present slopes; in the VLMs, the pattern is reversed. Enumeration remains accurate past the point where humans lose count, suggesting a form of subitizing that scales differently. Most strikingly, a reasoning model with adaptive deliberation declines to deliberate on detection tasks altogether, meaning a single search expresses itself as an effort gradient in one model and as an accuracy cliff in another. Wick argues these behavioral divergences are as informative as the agreements, providing a sharp and inexpensive probe of machine visual cognition that reveals where VLMs genuinely see like us—and where they do something entirely different.