HPG algorithm slashes embedding routing regret to O~(s√(MT))
New method avoids curse of dimensionality in dynamic model selection.
Deep Dive
Yan Dai, Negin Golrezaei, and Patrick Jaillet propose Hypentropy Policy Gradient (HPG) for routing queries to multiple embedding models in recommendation systems. The algorithm achieves $\tilde{\mathcal O}(s\sqrt{M T})$ linearized policy regret under adversarial conditions and bandit feedback, where $s$ is the intrinsic rank of experts, $M$ is the number of models, and $T$ is the number of rounds. It uses a log-quadratic policy class to capture query-dependent routing efficiently, with a computationally efficient, parameter-free implementation.
Key Points
- Problem formalized as adversarial contextual linear bandit with low-rank experts, enabling theoretical analysis.
- HPG achieves regret bound O~(s√(M T)) where s is intrinsic rank, M number of models, T rounds — avoiding exponential dependence on M.
- Algorithm is parameter-free and computationally efficient, suitable for real-world deployment.
Why It Matters
Practical, optimal routing for embedding models in recommendations could reduce latency and improve relevance at scale.