CCBENCH Finds Top LLMs Score Only 20-30% on Cultural Health Norms

A new research paper introduces CCBENCH, a framework for assessing large language models' (LLMs) cultural competence by evaluating how well they infer and adapt to a user's implicitly signaled cultural values. Rather than treating culture as a binary belonging state, CCBENCH models it as a continuum of norm-adherence. As a case study, the authors created CCBENCH-Health, comprising 60 theoretically grounded personas across six cultures, each engaging in 18 realistic dialogues. Each persona was evaluated on 52 authentic healthcare questions drawn from real user forums, yielding 3,120 unique interactions.

Benchmarking five leading LLMs revealed that even the best models produced culturally appropriate responses only 20-30% of the time. Explicitly prompting models to focus on cultural cues via chain-of-thought reasoning improved performance by just 3-5%. Notably, models performed better when personas avoided cultural norms rather than followed them, suggesting a built-in bias toward default Western values. The Afghan cultural context was the worst, with only 8.8% appropriate responses. Models also adapted more readily to implicit conversational styles than explicitly stated practices, highlighting a critical gap in cultural AI alignment.