From Raw IDs to Semantic Planning: Recommender Systems' Next Leap

A new paper from researchers at multiple institutions (Jin et al.) traces the evolution of recommender systems over two decades. Initially, industrial systems relied on raw IDs—globally unique, discrete identifiers with no inherent meaning—that enabled exact item lookup and memorization at massive scale. However, these IDs are semantically opaque, limiting personalization. Over time, systems began incorporating richer information: item content, context, multimodal signals, and cross-domain structure. This led to semantic IDs, which encapsulate that information into a more structured, model-facing identity rather than treating it as auxiliary features. The paper argues this is a broader shift beyond generative recommendation.

The paper then introduces 'semantic planning' as a future stage. Instead of retrieving items directly, the system would first predict the semantic target of the next user exposure (e.g., a category or concept) and then instantiate that target as a specific item or even a generated creative. This approach could fundamentally change model architecture, evaluation metrics, and how recommender systems coordinate the often-conflicting objectives of users, platforms, and content providers. The authors suggest this shift mirrors the progression from raw IDs to semantic IDs and may require new ways of thinking about recommendation accuracy and fairness.