ERICA method quantifies cluster replicability, reveals risks in gene expression analysis

Clustering is ubiquitous in science and machine learning, but until now there has been no quantitative framework to assess whether discovered clusters are replicable. A new preprint from Stanford researchers introduces ERICA (Evaluating Replicability via Iterative Clustering Assignments), a pipeline that computes a statistic to determine whether structure found in a dataset is consistently reproducible. The method also provides quantitative visualizations to answer practical questions like how similar clusters are to one another and which points might be outliers.

In experiments with synthetic data, ERICA successfully identified clusters that were replicable. However, when applied to three real-world gene expression datasets used for breast cancer subtype validation, the pipeline flagged cases where cluster assignments were not replicable. This finding highlights a critical vulnerability in biomedical ML: commonly used clustering results may not be stable, potentially leading to incorrect subtype classifications. The authors position ERICA as a practical tool for any researcher needing to validate the robustness of their clustering analysis, with implications for fields from genomics to customer segmentation.