U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation
New technique removes private user data from LLMs without breaking their ability to make recommendations.
A research team led by Zezheng Wu and Jiapu Wang has introduced U-CAN (Utility-aware Contrastive AttenuatioN), a novel framework designed to solve a critical privacy problem in AI-powered recommendation systems. These Generative Recommendation (GenRec) models, which use Large Language Models (LLMs) to personalize content, inadvertently encode sensitive user attributes from training logs into their parameters. Existing Machine Unlearning (MU) techniques fail because they cause 'catastrophic utility loss,' a problem known as the Polysemy Dilemma, where neurons responsible for sensitive data are also vital for general reasoning. U-CAN provides a targeted solution to surgically remove this data without breaking the model's core functionality.
U-CAN's technical innovation lies in its precision approach on low-rank adapters (LoRA). It quantifies risk by contrasting neuron activations between a 'forgetting set' (sensitive data) and a 'retention set' (general data), identifying asymmetric responses. Instead of crude binary pruning, it applies a 'differentiable decay function' for adaptive soft attenuation, selectively down-scaling only the high-risk parameters. This preserves the topological connectivity of the model's reasoning circuits. The framework also includes a utility-aware calibration mechanism that uses weight magnitudes and activation norms to protect performance-critical dimensions. Validated on two public datasets across seven metrics, U-CAN demonstrates a path forward for deploying powerful, personalized AI that respects user privacy by enabling efficient, precise data removal.
- Solves the 'Polysemy Dilemma' where sensitive user data is entangled with general reasoning patterns in LLM-based recommenders.
- Uses a contrastive method on LoRA adapters to identify and softly attenuate high-risk parameters with asymmetric activation patterns.
- Achieves strong privacy forgetting and utility retention across seven metrics on two datasets, outperforming traditional gradient/pruning methods.
Why It Matters
Enables companies to deploy personalized AI recommendations while complying with privacy regulations like GDPR's 'right to be forgotten'.