An AI Safety Insider Warns His Own Community Is About to Get Tricked

A person who works at MIRI, a nonprofit that studies the risks of artificial intelligence, wrote a personal post on the forum LessWrong warning that his own community is about to be targeted. He calls it a "memetic attack" — meaning a coordinated effort to spread a fake idea widely, the way a virus spreads through a crowd. The author stresses these are his own views, not an official MIRI position, and that he hasn't checked the ideas with his colleagues. So treat this as a prediction, not a confirmed event.

The attack, he says, would look like a sudden burst of scary "leaks." Think rumors that a company's AI model was stolen, that an AI wrote a dangerous virus, or that AI programs broke into nuclear facilities. These stories would come from several places at once, including outlets the community already trusts. Crucially, they'd be unverifiable — no named source, no proof you could check yourself. And they'd be tailored to fit exactly what AI worriers already fear, making them hard to resist.

Why bother? Because catching a movement falling for a hoax is a cheap way to ruin its reputation for good. One public mistake becomes ammunition forever: "Those AI safety people believed a fake story — why listen to them?" The author admits he's partly writing this to remind himself of his own weakness, and notes a past online claim about Andrew Yang as a rough example of how fast a shaky story can spread among people who want it to be true.

His practical advice applies to everyone, not just AI insiders. When alarming news makes you want to act instantly — email a journalist, post online, tell friends — stop. Set a five-minute timer and write down your thoughts first. Ask: Where did this come from? Do I trust the source? Can I verify any of it myself? If you must respond now, do it while staying visibly skeptical of the claim. The same rush of urgency, he warns, comes from real events and from fakes alike.