GMM+LLM method boosts clustering for underrepresented topics
Unsupervised augmentation with GMM-LLM preserves quality and enriches minority clusters.
A new paper from Noor Khalal, Abdallah Alaa-Eddine Djamai, Imed Keraghel, and Mohamed Nadif tackles a persistent problem in unsupervised NLP: underrepresented topics get lost during clustering. The researchers introduce a targeted data augmentation pipeline that pairs Gaussian Mixture Models (GMMs) with Large Language Models (LLMs). GMMs first identify clusters where data is sparse or imbalanced, then LLMs generate synthetic text documents to fill those gaps and balance the data.
Experiments across various imbalanced text datasets show that this approach preserves clustering performance in all cases, while often making clusters more interpretable. This matters because traditional augmentation often dilutes real signal, but the GMM-LLM approach targets only the minority clusters, avoiding unnecessary noise. The method is fully unsupervised, making it a practical fit for real-world NLP workloads where labeled data is scarce. Published at IDA 2025 and available on arXiv (arXiv:2607.28635), the method offers a robust, scalable route to better data representation in unsupervised tasks.
- Uses GMMs to detect minority clusters, then LLMs to generate synthetic data for those areas
- Preserves clustering performance across all imbalanced datasets tested, often boosting interpretability
- Fully unsupervised approach from IDA 2025 paper, arXiv:2607.28635
Why It Matters
Improves topic clustering in NLP without labels, enabling better analysis of rare but critical topics.