AI Read 154,000 Websites to Grade Who's Actually Green

Companies and nonprofits love to say they care about the environment, but that information is scattered across dozens of messy webpages, PDFs and newsletters. No human can read all of it. So researchers built a system that uses AI language models — software that reads and understands ordinary text — to scan entire websites and pull out concrete environmental actions, like installing solar panels or cutting waste. They tested it on 4,964 American Jewish congregations, successfully crawling 2,657 of their websites and collecting 154,454 webpages.

The team compared three ways of doing this. One searched for green keywords first, then asked the AI to judge the results. One used meaning-based search (finding pages similar in topic, not just wording), then asked the AI. The third just handed everything to the AI. A human expert checked the AI's answers, and agreement was weakest for keyword search, better for meaning-based search, and roughly equal for letting the AI read everything. So the shortcut that saves the most computer power also threw away real information — the meaning-based filter missed about 13 percent of relevant content before the AI ever saw it.

Here's the catch. Even the best method only moderately agreed with the human expert, meaning plenty of judgment calls were wrong in one direction or another. When the AI read everything unfiltered, it flagged green activity at 1,398 congregations — 53 percent of those with working sites — which is more coverage but also more chance of false positives. A church that merely mentions climate change in a sermon could be counted as taking action. And a poorly designed or rarely updated website makes an organization look inactive even if it isn't.

Why should you care? Green claims are everywhere, and verifying them is expensive and slow. This research shows AI can do the first pass cheaply, at a scale no team of analysts could match, and the method is designed to be reused on other kinds of institutions — companies, universities, city governments. That's the promise of real accountability for greenwashing. It also means the tools that grade your employer's environmental record may soon be automated, imperfect, and very hard to argue with.