arXiv study: AI confuses olives with vehicles in concept alignment test

A new arXiv paper by Sunayana Rane, Brenden M. Lake, and Thomas L. Griffiths investigates how AI systems understand everyday concepts by asking about implausible category members. Instead of typical questions like 'Is a car a vehicle?', they asked 'Is an olive a vehicle?' to probe the edges of conceptual boundaries. Using stimuli from the classic Rosch and Mervis psychological study, they compared AI and human assignments of objects to superordinate categories—both correct and mismatched categories. The method reveals whether AI truly grasps concept structure or merely recalls training data patterns.

The results show systematic concept misalignment: AI systems treat 'words' as belonging to 'vehicles' and 'clothing' categories, misclassify several 'vegetable' members as 'fruit', and assign non-weapon exemplars to the 'weapons' category. These errors differ significantly from human responses. The authors demonstrate that these misalignments lead to problematic downstream behavior with direct implications for AI safety—such as unsafe tool use or biased decisions. The paper underscores that standard concept probing is insufficient and that testing with implausible members is a more rigorous method for evaluating AI concept understanding.