RouteRec shows item-level aggregation beats single-agent routing for recommenders
Request-level selection of recommender agents found too coarse for sparse settings—learned item-level aggregation wins.
Researchers from Kaiji Zhou's team propose RouteRec, a strict evaluation framework for recommender-agent selection and aggregation. The study pits four traditional agents (collaborative filters, sequential models, content-based retrievers) plus an LLM reranker against each other on MovieLens-1M. Using a leakage-free 5-fold out-of-fold protocol, they compared request-level hard selection with item-level learned aggregation. The full quality oracle showed substantial headroom (HR@10 = 0.584), confirming useful cross-agent signal exists—but hard selection alone fell below the simple BM25 baseline, even when selectively escalating to the LLM.
Item-level aggregation told a different story. The cheap-only learned aggregation variant matched BM25 in HR@10 while posting a higher NDCG (0.123 vs 0.114). Gated all-agent aggregation achieved HR@10 = 0.295, albeit with 70.2% of queries using the LLM reranker. The conclusion is clear: picking one recommender's entire output list per request is too coarse for sparse fixed-candidate settings. Instead, combining items from multiple agents at a per-item level unlocks significantly better performance, paving the way for more sophisticated routing strategies in production recommender systems.
- RouteRec compares request-level hard selection vs item-level learned aggregation across 4 traditional recommenders + 1 LLM reranker on MovieLens-1M
- Hard selection underperforms BM25 baseline (0.223 vs 0.254 HR@10), and selective LLM escalation fails to improve it
- Learned item-level aggregation reaches HR@10 = 0.295 with 70.2% LLM calls, proving per-item mixing outperforms whole-list routing
Why It Matters
Recommender systems can now precisely combine agent outputs per item, dramatically improving relevance without ballooning costs.