BACH: New Bayesian model solves routing collapse in multi-interest retrieval
Soft mixture training gives each user a personalized interest weight vector at serving.
Two-tower retrievers have long been the backbone of large-scale recommendation systems, but their standard single-embedding user representation limits the ability to capture diverse interests. Multi-interest models address this by assigning each user multiple heads, but they typically rely on hard-routing training (each item assigned to one head), which leads to head under-utilization (routing collapse) and fails to give per-user interest importance estimates.
Now, a team of researchers proposes BACH (Bayesian Admixture of Contrastive Heads), reframing multi-interest retrieval as a per-user Bayesian mixture over heads. By applying variational inference, the model softly allocates each training example across all heads, effectively eliminating routing collapse. The learned per-user mixture weights are then reused at serving time without additional computation. BACH also introduces a shared global-codebook variant that supports precomputable retrieval. Evaluated on three large-scale benchmarks (MovieLens-20M, Taobao, Netflix), BACH consistently outperforms both hard-routing multi-interest models and single-vector baselines, especially in top-of-ranking metrics—a critical improvement for real-world recommendation systems where only the first few results matter.
- BACH uses variational inference to softly assign each user's interests across multiple heads, preventing the head collapse common in hard-routing models.
- The model produces per-user interest mixture weights that are reused at serving time, requiring no additional inference cost.
- On MovieLens-20M, Taobao, and Netflix benchmarks, BACH improves top-of-ranking retrieval over existing baselines at every head count tested.
Why It Matters
BACH enables recommendation systems to capture diverse user interests accurately while avoiding head under-utilization, improving real-world retrieval quality.