MOSAIC: Unveiling the Moral, Social and Individual Dimensions of Large Language Models

Researchers Erica Coppolillo and Emilio Ferrara have unveiled MOSAIC, a groundbreaking benchmark designed to comprehensively evaluate the ethical foundations of large language models (LLMs). As AI systems like GPT-4 and Claude 3 are increasingly deployed in sensitive domains such as psychological support and high-stakes decision-making, existing evaluation methods have relied almost exclusively on Moral Foundation Theory (MFT), neglecting crucial dimensions like social values and individual personality traits. MOSAIC addresses this critical gap by creating the first framework that jointly assesses moral, social, and individual characteristics, providing a more holistic view of AI ethical reasoning.

The benchmark comprises nine validated questionnaires drawn from moral philosophy, psychology, and social theory, alongside four platform-based games designed to probe morally ambiguous scenarios. With over 600 curated questions and scenarios, MOSAIC has been validated across three different LLM families, demonstrating that MFT alone is insufficient for comprehensive ethical evaluation. The researchers have publicly released both the dataset and a Python library, enabling developers and researchers to test their models against this new standard. This represents a significant advancement in AI safety research, moving beyond simplistic moral frameworks toward understanding how AI systems navigate complex human ethical landscapes.