Position-Aware Sequential Attention for Accurate Next Item Recommendations

Researchers Timur Nabiev and Evgeny Frolov have published a new paper, 'Position-Aware Sequential Attention for Accurate Next Item Recommendations,' proposing a fundamental shift in how sequential models like those used by Netflix or Amazon understand order. The paper argues that the standard method—injecting positional information via additive embeddings—is flawed because it entangles positional signals with item semantics, causing this crucial timing data to propagate weakly in deep neural networks and limiting the model's ability to capture complex sequential patterns. This weakness makes next-item prediction, a core task for recommendation engines, less accurate than it could be.

The team's solution is a kernelized self-attention mechanism where a dedicated, learnable 'positional kernel' operates separately from the semantic similarity calculations. This kernel directly modulates the attention weights within each block, cleanly disentangling 'when' something happened from 'what' it was. This architecture enables adaptive, multi-scale modeling of sequences. In practical terms, this means a streaming service could more accurately predict the next song in a workout playlist, or an e-commerce platform could better anticipate a user's next purchase based on their browsing history. The model has already demonstrated consistent performance improvements over established baselines in experiments, pointing toward more precise and efficient recommendation systems.