ChatGPT making up words

A viral Reddit post has highlighted a persistent and critical flaw in large language models like OpenAI's ChatGPT: their tendency to hallucinate. A user in the UK sought advice on a property law matter involving a 'useless freeholder.' In its response, ChatGPT confidently stated the user's case was 'strongening'—a nonsensical, invented word that blends 'strengthening' and 'strong.' This is not a simple typo but a fabrication, demonstrating the model's ability to generate coherent yet entirely incorrect content.

This specific error occurred in the complex domain of UK property law, a field where precise terminology is paramount. The incident underscores that while models like GPT-4 are powerful tools for drafting and brainstorming, they lack true understanding and can produce authoritative-sounding falsehoods. For professionals, this serves as a critical reminder that AI outputs, especially for legal, medical, or financial advice, must be rigorously fact-checked by a human expert. The core technology, including techniques like Reinforcement Learning from Human Feedback (RLHF), has not yet solved the hallucination problem.

The 'strongening' case is a microcosm of a broader challenge. As businesses and individuals increasingly integrate AI agents into workflows that require accuracy—from contract review to customer support—these hallucinations pose tangible risks of misinformation and poor decision-making. It reinforces the necessity of using AI as an assistive tool with a human in the loop, rather than as an autonomous source of truth for specialized knowledge.