IBA framework rethinks latent reasoning allocation for generative recommendations
Earlier semantic ID tokens matter more – allocate reasoning steps there for better accuracy.
A new paper from Shangxin Yang, Min Gao, Zongwei Wang, and Junliang Yu tackles a core efficiency problem in semantic ID-based generative recommendation. In this paradigm, items are represented by a short sequence of semantic ID tokens generated autoregressively. Recent work introduced latent reasoning—extra hidden-state computations before each token decision—to improve quality. But a practical question remained: should every token position receive the same number of refinement steps, or can we allocate them more intelligently?
The authors analyze position-wise information gain (IG), measuring how much each ID token reduces uncertainty about the target item. They observe that earlier positions consistently yield higher IG, while later ones add diminishing returns. Based on this, they design IBA (Information-Gain Budget Allocation), a framework that treats refinement steps as a limited computational resource and learns to allocate more steps to high-IG positions. Experiments across multiple public datasets show that IBA beats strong baselines and achieves a superior accuracy–computation trade-off compared to fixed or naïve allocation strategies.
- IBA allocates more latent reasoning steps to early semantic ID tokens with higher information gain.
- Paper reports consistent improvements over fixed-step baselines across multiple public recommendation datasets.
- Achieves a better accuracy–computation trade-off without increasing total reasoning budget.
Why It Matters
Smarter reasoning step allocation could make generative recommenders faster and more accurate, saving computational resources.