Ask AI for Software Picks? It Often Cites Spam Sites, Study Finds

If you've ever asked an AI chatbot which software to buy, you probably assumed it was reading trusted reviews. A study put that to the test. Researchers asked Perplexity, an AI search tool, for top recommendations in 380 software categories like "CRM software" and "museum management software." They then looked at every website Perplexity cited as its source. The results were unsettling: nearly 60% of the 7,534 citations pointed to websites that rank below the 100,000th most popular site on the internet. A quarter weren't even in the top million.

Where does the rest come from? Some of it is legitimate — Reddit and Gartner appear frequently. But the third most-cited source was Guideflow, a company's marketing blog that sells product demo software. It isn't a review site or an expert publisher, yet Perplexity cited it 194 times across 96 categories. That means a vendor's own articles, written to attract customers, helped shape AI buying advice.

The researchers also found a web of content farms. Three sites — wifitalents.com, worldmetrics.org, and gitnux.org — appear to be run by the same group. Together, they published over 215,000 machine-generated "best [product]" pages. All three were created in late 2023 or later. None had a history of quality or authority. Yet AI treated them as reliable evidence.

What does this mean for you? AI is convenient, but it's not a neutral librarian. It repeats whatever it finds, including spammy, self-serving content. The catch is you can't easily tell which sources it used. Until AI companies make their citations more transparent, treat AI recommendations as a starting point — not the final word.