ACE fixes LLM embedding anisotropy, boosting recommendation recall by 12.4%
LLM embeddings are too clustered; ACE reshapes them for better recommendations.
A team of researchers from South Korea (Dongcheol Lee, Hye-young Kim, Jongwuk Lee) has introduced ACE (Anisotropy-Controllable Embedding), a method that addresses a critical limitation in the LLM-as-Extractor paradigm for sequential recommendation. Large language models produce semantically rich item embeddings, but these embeddings often exhibit strong anisotropy—meaning most vectors cluster in similar directions. This geometric imbalance makes it difficult for downstream sequential recommendation models to adapt collaborative signals during fine-tuning, ultimately hurting performance.
ACE solves this by applying a linear autoencoder (LAE) that reshapes the embedding distribution while preserving its semantic structure. The key innovation is the use of an L2-regularization term that controls the dispersion of embedding dimensions, effectively reducing anisotropy, while a reconstruction loss maintains the semantic relationships among items. This balance between geometric uniformity and semantic preservation enables more stable and effective learning. In extensive experiments, ACE consistently outperformed existing LLM-enhanced sequential recommendation models, achieving improvements of up to 12.4% in Recall@20 and 11.8% in NDCG@20. The paper has been accepted at SIGIR 2026.
- LLM-generated embeddings in recommendation systems suffer from anisotropy, clustering vectors in similar directions and weakening collaborative signals.
- ACE uses a linear autoencoder with L2 regularization to spread embeddings while preserving semantic relationships via reconstruction loss.
- Achieves up to 12.4% improvement in Recall@20 and 11.8% improvement in NDCG@20 over existing LLM-enhanced SR models.
Why It Matters
Makes LLM-based recommendations more accurate by fixing a fundamental geometric flaw in embeddings.