Shopee's KGD boosts recommender accuracy 12% with decoupled knowledge transfer

Industrial recommender systems increasingly rely on pretrain-then-transfer, but continuous streaming data causes behavioral distribution drift, making pretrained models stale. Shopee's research team, in a new arXiv paper, introduces Knowledge-Geometry Decoupling (KGD) to solve two key questions: what to learn from user behavior sequences, and how to transfer that knowledge when the pretrained model is periodically refreshed.

KGD addresses the first question with Behavioral Multi-Token Prediction (BMTP), which replaces conventional next-token prediction. Instead of treating every adjacent click as a dependency, BMTP retains only collaboratively or semantically related future items as supervision. This eliminates spurious transitions across unrelated sessions, producing cleaner and more transferable behavioral knowledge. For the transfer problem, KGD assigns knowledge and task geometry to separate parameter sets: a refreshable encoder owns the behavioral knowledge, while a task learner reads the encoder states through read-only cross-attention and writes task-specific geometry through a mechanism called Anchored Calibration Residual (ACR), which stays orthogonal to the pretrained embedding. This decoupling prevents task-gradient interference during knowledge updates and allows continual refresh without invalidating downstream adaptation.

Experiments show KGD improves over strong pretrain-transfer baselines by 4–12% on eight public benchmarks and maintains its edge over a 90-day production stream where baselines plateau. KGD is fully deployed at Shopee. In a live A/B test on Shopee's homepage search, it increased gross merchandise value per user by 1.75% and advertising revenue by 1.53%, proving the method's practical value at scale. The team also released core implementation code, inviting further adoption.