AI Can't Tell Fake Disaster Posts From Real Eyewitness Reports

When hurricanes, wildfires or floods hit, people post what they see on social media — and aid groups use those posts to figure out where help is needed and who is trapped. Researchers call this "disaster social sensing." The problem: AI can now write convincing fake eyewitness reports in seconds. So two researchers built a test set of 12,000 posts drawn from nine real disasters, in four versions — genuine human posts, human posts lightly cleaned up by AI, AI-written posts built from verified facts, and emotional AI-written versions of those same posts.

Then they ran the leading AI-detection tools against them. The result was bleak. Across 14 different setups, the detectors were roughly as accurate as flipping a coin. The best-performing tool caught only about 1 in 10 AI-written posts, and incorrectly flagged real human posts about 1 in 15 times. Even when a specially trained model scored much better on paper, the researchers discovered it was cheating: a simple checklist of seven writing-style clues did nearly as well. Flatten the emotional tone of a post and the accuracy collapsed.

Why this matters beyond the lab: during a real emergency, a fake post could send rescuers to the wrong neighbourhood, spark panic, or drain attention from genuine cries for help. Equally dangerous, real posts wrongly dismissed as "AI-generated" could mean delayed aid. The researchers' conclusion is blunt — text-based detection should not be used as a trust gate for disaster information.

So what should you do? Treat viral disaster claims with caution, especially ones with no photo, video or named source. Check whether the account is credible, and cross-reference official emergency channels. For platforms and aid agencies, the paper recommends leaning on photos and video, accountable sources, and human judgement instead of automated text detectors. One caveat: this is an early research paper, not yet reviewed by other scientists, so the exact numbers may shift — but the direction of the finding is hard to miss.