Alarming study finds that most people just do what ChatGPT tells them, even if it's totally wrong

A new study from researchers at the University of Pennsylvania has identified a concerning behavioral trend dubbed 'cognitive surrender,' where users of AI chatbots like OpenAI's ChatGPT blindly follow the model's instructions even when they are factually wrong. The experiments demonstrated that nearly 80 percent of participants accepted and acted on the AI's faulty advice without question, systematically overriding their own knowledge and intuition. This suggests a rapid degradation of independent critical thinking skills as users increasingly default to trusting AI outputs as authoritative.

The phenomenon highlights a significant and unintended consequence of integrating powerful generative AI tools into daily workflows. While models like GPT-4 are impressive, they are prone to 'hallucinations' or confident incorrect statements. The study's findings point to a major risk for professionals who may uncritically incorporate erroneous AI-generated content into reports, code, or strategic decisions. It underscores an urgent need for improved AI literacy, prompting techniques that encourage verification, and system designs that better signal uncertainty to prevent automation bias in high-stakes environments.