New paper argues LLMs cannot be true thinking partners, confirming 'Innovation Illusion'

A new paper by S.F.M. van Vlijmen and H.D. Lethe jr. (arXiv:2606.07722) offers a provocative take on large language models (LLMs) and chatbots. Spanning 42 pages with 3 figures, the work combines insights from Aggregation Dynamics, Cognitive Linguistics, Neuropsychology, and Psychology to examine what chatbots can and cannot do in problem-solving conversations. The authors distinguish basic chatbots (a simple LLM with an interface) from more advanced systems, aiming to identify core limitations.

The central argument introduces 'metaphorical problem propagations' as a model of human understanding, then hypothesizes that training datasets only partially replicate this process, leading to artificial propagations encoded in LLM weights. Crucially, the paper concludes that basic chatbots cannot serve as genuine thinking partners capable of matching human cognition, and that further scaling of LLMs will not overcome this fundamental gap. The authors cite Yann LeCun's statement that animals and humans exhibit far superior learning and understanding, and argue that Big Tech's optimism about AGI is misplaced. Despite this, they acknowledge the widespread adoption of chatbots and call for a realistic social and political understanding of their actual benefits and drawbacks.