CL-bench Life: Can Language Models Learn from Real-Life Context?

A new paper by Shihan Dou and 37 other researchers presents CL-bench Life, a fully human-curated benchmark designed to test whether language models can learn from real-life contexts. Unlike existing benchmarks that focus on clean, professional scenarios, CL-bench Life includes 405 context-task pairs and 5,348 verification rubrics drawn from messy, fragmented everyday situations—multi-party chats, personal archives, and behavioral traces. The researchers evaluated 10 frontier language models (including OpenAI's GPT-4, Anthropic's Claude, and others) and found that real-life context learning remains a major challenge. The best-performing model solved only 19.3% of tasks, with an average of 13.8% across all models. Models particularly struggled with reasoning over noisy group chat histories and incomplete behavioral records.

The poor performance reveals a critical gap in current AI assistants. While models excel at structured tasks like coding or document analysis, they fail to handle the ambiguous, socially embedded contexts that humans navigate daily. CL-bench Life provides a crucial testbed for advancing this capability—progress on it could lead to more intelligent, reliable assistants that truly understand users' lives. The authors emphasize that as AI assistants move from professional settings into everyday life, improving context learning is essential for tasks like managing personal schedules, interpreting conversations, or making sense of scattered notes. The benchmark is open-source and aims to drive research toward models that can learn in the wild.