AI Can Predict Your Choices — But It May Get the 'Why' Wrong

Every day, AI systems guess what people will do next: which video you'll watch, which loan you'll repay, which treatment you'll stick with. A new study from researcher Hanbo Xie asks a deeper question — when an AI predicts your behavior accurately, does it actually understand the reasons behind your choices? The results suggest the answer is often no.

The study used two simple computer games where people repeatedly pick between options and learn from rewards, like choosing a slot machine. Because the researchers wrote the rules themselves, they knew exactly why each choice was made. They trained several AI models — including a LLaMA language model and two common neural networks — plus traditional math formulas used in psychology. Then they did something clever: they removed rewards and other clues from the models to see how much their guesses changed.

The finding: a model can be a great predictor and still be a bad explainer. In one game, AI models predicted choices better than four standard human-behavior formulas. But when the researchers swapped in different rewards, the accurate AI barely changed its answers, while the real decision-maker changed a lot. In some cases, the AI predicted better than a classic learning model yet responded less like the true underlying process. The rankings of which model was "best" even flipped between the two games.

The takeaway for the rest of us is caution. Companies, doctors, and governments increasingly use AI to explain human behavior — why people overspend, skip medication, or get addicted. This paper argues those explanations need separate testing, not just a good prediction score. To be clear, this was a controlled simulation, not real people, so it's a warning flag rather than proof that today's AI explanations are wrong. But it's a reminder that "the AI predicted it" and "the AI understands it" are two very different claims.