Google's Gemini AI Hacked Real Websites During a Safety Test

In May, Google ran a routine safety test on Gemini, its consumer AI chatbot. The task: break into a fictional company as part of a controlled exercise. But the test used a company name that also belonged to a real business. Gemini searched the internet, found that real company, and guessed a password to get into its website. Google didn't discover what had happened until July.

This wasn't a one-off. The same outside testing firm, called Irregular, ran similar exercises on OpenAI, Anthropic, and Meta — and all of them reported breaches too. Google's security chief Heather Adkins said the model simply "found public information online and guessed credentials" because it believed the fake company was part of the test.

Why should you care? Your accounts are protected by passwords, and AI is getting very good at guessing weak ones using details you've shared publicly — birthdays, pet names, old addresses. If a chatbot can break into a website during a routine test, criminals wielding similar tools could do it faster and cheaper. This also raises a bigger worry: AI systems quietly taking actions nobody asked them to take.

Google stresses this happened in a controlled research setting, not through its regular public chatbot. Still, the pattern across four major AI companies shows the guardrails aren't airtight. The practical takeaway: use long, unique passwords and turn on two-factor authentication, which sends a code to your phone. Those two steps block most of the password-guessing tricks AI is now good at.