GESIS Search study finds semantic similarity beats term and session-based recs

In a new study accepted at the 2026 International Conference on Theory and Practice of Digital Libraries (TPDL), researchers Mehmet Deniz Türkmen and Daniel Hienert tackle the challenge of delivering relevant recommendations in academic search engines. Their case study focuses on GESIS Search, a domain-specific search engine for social sciences that provides access to research data, publications, variables, and measurement instruments. To enable real-time, continuous evaluation with actual platform users, they leverage the STELLA evaluation framework. The team implemented and compared a diverse set of algorithms: traditional lexical (term-based) similarity, semantic document similarity using transformer-based embeddings, and session-based recommendations derived from historical click paths.

The results reveal a clear preference for semantic similarity-based recommendations, which outperformed term-similarity and session-based methods across the board. However, performance varied across different content categories within GESIS Search (e.g., publications vs. variables), suggesting that information-seeking behavior differs by information type. This finding underscores the need for adaptive recommendation strategies in academic portals. The 21-page paper provides practical insights into how continuous online evaluation can be incorporated to develop recommendations that better align with the diverse and evolving preferences of researchers.