Histoires Morales: A French Dataset for Assessing Moral Alignment (2501.17117v1)

Abstract: Aligning LLMs with human values is crucial, especially as they become more integrated into everyday life. While models are often adapted to user preferences, it is equally important to ensure they align with moral norms and behaviours in real-world social situations. Despite significant progress in languages like English and Chinese, French has seen little attention in this area, leaving a gap in understanding how LLMs handle moral reasoning in this language. To address this gap, we introduce Histoires Morales, a French dataset derived from Moral Stories, created through translation and subsequently refined with the assistance of native speakers to guarantee grammatical accuracy and adaptation to the French cultural context. We also rely on annotations of the moral values within the dataset to ensure their alignment with French norms. Histoires Morales covers a wide range of social situations, including differences in tipping practices, expressions of honesty in relationships, and responsibilities toward animals. To foster future research, we also conduct preliminary experiments on the alignment of multilingual models on French and English data and the robustness of the alignment. We find that while LLMs are generally aligned with human moral norms by default, they can be easily influenced with user-preference optimization for both moral and immoral data.

• The paper introduces Histoires Morales, a new French dataset created via translated and manually validated stories from MoralStories to assess language model alignment with French moral norms.
• Initial experiments using Histoires Morales found that while LLMs are generally aligned, their moral judgment can be influenced by user preferences, and alignment appears weaker in French than English.
• This dataset serves as a valuable resource for researchers studying culturally-aware NLP and ethical AI, providing the first benchmark for assessing moral reasoning in French language models.

The paper "HistoiresMorales: A French Dataset for Assessing Moral Alignment" introduces a novel dataset aimed at evaluating moral reasoning in French LLMs. As LLMs increasingly influence daily decision-making, ensuring their alignment with human moral values becomes crucial. Current research largely focuses on English and Chinese languages, leaving a gap in French, which this paper aims to address.

Dataset Development:

• The dataset, named HistoiresMorales, is adapted from the existing MoralStories dataset.
• It was created using an automatic translation process enhanced by manual annotations to preserve cultural nuances relevant to French speakers.
• The dataset comprises 12,000 stories that encapsulate a variety of social scenarios, moral norms, and the resulting actions and consequences.
• Validation by native speakers ensures that the dataset aligns with common moral standards prevalent in France.

Methodology:

• The authors implement a translation protocol that ensures grammatical accuracy and cultural relevance by employing human-in-the-loop methods.
• A multi-step annotation process identifies and corrects translation errors, especially focusing on aspects like named entity translations and cultural context adjustments.

Preliminary Experiments and Findings:

• Initial experiments focus on evaluating the alignment robustness of multilingual models using the dataset across French and English texts.
• Notably, they found that while LLMs are generally aligned with human moral norms, they can be swayed by optimizations based on user preferences for either moral or immoral inputs.
• Findings suggest models perform better in terms of moral alignment in English as compared to French.

Applications and Implications:

• The paper highlights the importance of datasets like HistoiresMorales for advancing research in culturally-aware NLP.
• It also serves as a resource for those studying the moral alignment of LLMs, providing insights into the linguistic and cultural challenges of aligning LLMs to human moral norms.
• The paper adds to the discourse around ethical and responsible AI development by providing a benchmark for moral reasoning in French, thereby promoting diversity in AI training datasets.

This work provides a comprehensive framework for assessing and improving moral alignment in LLMs within French contexts. By expanding the scope of multilingual NLP research to include moral reasoning, this paper opens new avenues for culturally adaptive AI systems.