Don't Measure Once: Measuring Visibility in AI Search (GEO)
AI search results vary across runs, making single measurements unreliable for assessing visibility.
A new research paper from Julius Schulte, Malte Bleeker, and Philipp Kaufmann challenges traditional approaches to measuring search visibility in the age of AI. Titled 'Don't Measure Once: Measuring Visibility in AI Search (GEO),' the study argues that the probabilistic nature of large language model-based chat systems makes single-query measurements unreliable. Unlike classical search engines where results are transparent and stable, AI search outputs can vary significantly across different runs, prompts, and time periods.
The researchers conducted empirical studies demonstrating that one-off observations fail to capture the true performance of Generative Engine Optimization (GEO) strategies. Their findings, presented in a 19-page paper with 7 figures and 17 tables, show that brands must adopt repeated measurement approaches to accurately assess their visibility in AI search systems. This represents a fundamental shift from treating visibility as a single-point outcome to characterizing it as a distribution, requiring new methodologies for SEO professionals and digital marketers working with AI-powered search tools.
- AI search results vary across runs, prompts, and time due to probabilistic LLM nature
- Traditional single-query SEO measurement fails for Generative Engine Optimization (GEO)
- Brands must use repeated measurements to characterize visibility as a distribution
Why It Matters
SEO professionals need new measurement strategies for AI search, as traditional approaches are now unreliable.