LUCid: Redefining Relevance For Lifelong Personalization

A team from the University of Michigan (Okite, Misra, Chai, Mihalcea) has released LUCid, a benchmark designed to measure how well AI systems handle lifelong personalization. The core problem is that current personalization engines operationalize relevance solely through semantic proximity—meaning they only surface user information from past interactions that are topically similar. LUCid tests whether models can retrieve situationally relevant context even when that context comes from semantically distant history. The benchmark comprises 1,936 realistic user queries paired with rich interaction histories spanning up to 500 sessions, creating a rigorous test of memory and contextual understanding.

Their experiments reveal a severe performance collapse across multiple architectures. On the hardest instances, retrieval recall dropped to near zero for all tested models, including state-of-the-art systems like Gemini-3-Flash, GPT-5.4, and Claude Haiku. Response alignment—how well the model's answer reflects the relevant user-specific context—remained at roughly 50%, barely above random chance. This exposes a fundamental mismatch: current models encode relevance as semantic similarity, but true personalization requires understanding user-centric relevance that can cut across topic boundaries. The implications are significant for both robustness and safety, as critical user attributes (e.g., health conditions, preferences, past decisions) may remain undetected simply because they appear in unrelated past conversations. LUCid provides a standardized way to evaluate and eventually correct this blind spot.