TriAlignGR: Triangular Multitask Alignment with Multimodal Deep Interest Mining for Generative Recommendation

TriAlignGR, developed by Yangchen Zeng and colleagues, addresses critical flaws in existing Semantic ID (SID) pipelines for generative recommendation. Traditional SID methods suffer from SID Content Degradation (SCD)—where cascaded encoding and residual quantization discard multimodal and interest-level semantics—and SID Semantic Opacity (SSO)—where models generate SID sequences without understanding their meaning, leading to hallucination and poor generalization. Prior work mostly handles text-SID alignment, ignoring visual semantics and latent user interests.

TriAlignGR solves these through three tightly integrated components: Cross-Modal Semantic Alignment (CMSA) integrates visual content into SID construction via VLM-generated textual descriptions and multimodal embeddings; Multimodal Deep Interest Mining (MDIM) uses LLM Chain-of-Thought reasoning to extract latent user intents (e.g., "productivity-focused lifestyle" from noise-canceling headphones); and Triangular Multitask (TMT) jointly trains eight complementary generation tasks—including two novel visual-semantic tasks (VisDesc→SID, VisDesc→Title)—under a single autoregressive loss, completing the SID-Text-Image triangle without task-specific towers or complex loss weighting.