MOSAIC: A New Benchmark for Assessing Moral, Social, and Individual Dimensions of LLMs
MOSAIC has been unveiled by researchers as the inaugural extensive benchmark aimed at assessing the moral, social, and individual traits of Large Language Models (LLMs). This benchmark fills a significant void in current research, which has predominantly focused on Moral Foundation Theory (MFT), often overlooking other essential aspects like social values, personality traits, and individual characteristics that influence ethical reasoning in humans. Comprising nine validated questionnaires from moral philosophy, psychology, and social theory, along with four interactive platform-based games to examine moral behavior, MOSAIC seeks to enhance the evaluation of LLMs' ethical reasoning abilities. This is crucial as these models are increasingly utilized in sensitive fields such as healthcare and psychological support. The related paper can be found on arXiv with the identifier 2603.00048, categorized as replace-cross. This work underscores the necessity for more comprehensive evaluation strategies in AI ethics research.
Key facts
- MOSAIC is the first large-scale benchmark to jointly assess moral, social, and individual characteristics of LLMs.
- Existing studies and benchmarks rely almost exclusively on Moral Foundation Theory (MFT).
- MOSAIC includes nine validated questionnaires from moral philosophy, psychology, and social theory.
- MOSAIC includes four platform-based games to probe moral behavior.
- LLMs are increasingly deployed in sensitive applications like psychological support, healthcare, and high-stakes decision-making.
- The paper is available on arXiv with identifier 2603.00048.
- The announcement type is replace-cross.
- The benchmark addresses limitations in current ethical reasoning evaluation.
Entities
Institutions
- arXiv