I don't understand AI. How does it work?

A simple question on Reddit—'How does AI formulate an answer to "How long should I boil spaghetti noodles?"'—sparked a detailed public explainer on how large language models (LLMs) like OpenAI's GPT-4, Anthropic's Claude, or Meta's Llama 3 actually work. User tlm11110's post questioned whether AI performs a web search to calculate an average, median, or mode of found answers. The thread's top responses clarified a fundamental distinction: modern LLMs are not search engines. They are prediction engines. Their core function is to generate the most probable next word (or 'token') in a sequence, based on patterns learned from ingesting terabytes of text data during training.

Instead of querying the live web, an LLM's knowledge is a frozen snapshot from its training data—corpora like Common Crawl, Wikipedia, and books. When asked about spaghetti, the model activates neural pathways associated with cooking instructions. It doesn't 'know' a fact but predicts a sequence like 'boil for 8 to 12 minutes' because that phrase, or its components, appeared with high frequency in relevant cooking contexts within its training set. The response is a complex statistical output, not a retrieved average. This process, called autoregressive generation, is why answers can vary subtly and sometimes 'hallucinate' if the predicted pattern leads to plausible-sounding but incorrect information.

Understanding this mechanism is crucial for using AI effectively. It explains the need for techniques like Retrieval-Augmented Generation (RAG), where models fetch current data from external sources to ground their predictions. It also highlights why prompt engineering matters—the words you provide set the initial context that steers the model's probability calculations toward a desired output format, like a step-by-step recipe versus a single number.