Publishers threaten to block Google's AI crawlers over traffic losses

According to a Wall Street Journal report, major publishers are increasingly considering cutting off Google's access to their content to escape the negative impact of Google's AI-powered search results (AI Overviews). The dilemma is stark: publishers can block Google's crawlers specifically for AI training (via the 'Google-Extended' bot), but doing so may still allow content to appear in AI Overviews. To fully prevent that, they would need to use the 'nosnippet' tag, which could reduce search visibility, or the more extreme 'noindex' tag, which removes them from search entirely. Reddit, which currently receives $60 million per year from Google for content scraping, has reportedly discussed shutting off access for AI use. Reuters President said they're weighing the economic trade-offs, USA Today's CEO declared 'enough is enough,' and the CEO of People Inc. confirmed blocking Google entirely is '100% on the table.' Politico and the Economist are also reconsidering their crawling permissions.

Google defended its AI features, stating they send 'billions of clicks to the web every week' and reiterated that it offers website owners clear controls to manage their content. The report highlights a growing tension between publishers who depend on search traffic and Google's shift toward providing AI-generated answers that reduce the need for users to click through to publisher sites. If a significant number of publishers adopt total blocking (via 'noindex'), it would represent a major escalation in the battle for content control in the AI era.