Harvard AI paper: Embeddings for preferences, not semantics

A new paper from Harvard researchers Carter Blair, Ariel Procaccia, and Milind Tambe tackles a fundamental flaw in using text embeddings for collective decision-making. When participants express opinions as free-form text, the natural approach is to embed those opinions in a vector space and apply facility location or fair clustering algorithms. However, standard text embeddings measure semantic similarity — which correlates poorly with preferential similarity, the true measure of agreement with a piece of text. The authors formalize this as an invariance problem: embedding models encode both preference-relevant signals (stance and values) and semantic nuisance (style and wording). Because these are observationally correlated in natural text, a geometry that relies on nuisance can appear preference-correct when it is not.

The solution involves synthetic training data designed to break the correlation between semantic and preferential similarity. This provably shifts the optimal scorer away from nuisance-dominated cosine and toward true preference representation. The method significantly improves preference prediction across 11 online deliberation datasets. This work has immediate implications for any system that uses text embeddings to aggregate opinions — from political deliberation platforms to survey analysis tools — and provides a mathematically principled way to separate what people say from how they say it.