Tabular embeddings need human-preference evaluation, says new paper
Current similarity rankings don't match human judgment, risking trust in PLM search.
A new position paper from Frederik Hoppe, Astrid Franz, Marianne Michaelis, Lars Kleinemeier, and Udo Göbel — accepted at the IJCAI/ECAI 2026 TRUST AI workshop — sounds the alarm on how tabular embeddings are evaluated for similarity search. The authors argue that leading embedding approaches (e.g., used in Product Lifecycle Management) are primarily optimized for prediction tasks rather than producing similarity rankings that align with human preferences. This disconnect, they claim, makes standard downstream metrics insufficient for assessing embedding trustworthiness in real-world business systems.
To address this, the paper presents a concrete evaluation procedure that measures human preference alignment. Using a PLM use case, they illustrate how current embeddings can fail to match expert judgments, potentially leading to poor search results and reduced trust. The authors call for a shift in evaluation practices to include human-aligned metrics, emphasizing that as tabular embeddings become more pervasive in enterprise applications, ensuring similarity search reflects actual user intent is critical for adoption and reliability.
- Current tabular embeddings are optimized for prediction, not human-aligned similarity rankings.
- Authors propose a new evaluation procedure to measure trustworthiness of similarity search.
- Product Lifecycle Management (PLM) use case reveals potential mismatches with expert preferences.
Why It Matters
As tabular embeddings power enterprise search, aligning with human judgment is essential for trust and accuracy.