WHALE unifies Wukong and HSTU for scalable recommendation modeling

WHALE tackles a fundamental challenge in industrial recommendation: unifying two complementary modeling paths. Wukong scales high-order non-sequence feature interactions (e.g., user demographics, item attributes, cross features) while HSTU scales long user-behavior sequence modeling. Prior architectures kept these paths separate. WHALE introduces layers where a Wukong module, an HSTU module, and an attention-based fusion module coexist. The fusion layer lets Wukong-derived interaction representations query HSTU-derived behavior representations, enabling progressive exchange of high-order feature crosses with fine-grained sequence evidence.

For practical deployment, the team designed custom Triton kernels and co-optimized the model-system interface to improve training and inference efficiency. On large-scale industrial data, WHALE achieves consistent offline improvements and positive online gains with only a modest serving-throughput trade-off. The architecture is already in production. This work provides a scalable blueprint for merging two critical sources of ranking signals, potentially influencing the next generation of recommendation systems.