---
title: Moral Dilemma Dataset (MDD) Overview
url: https://www.emergentmind.com/topics/moral-dilemma-dataset-mdd
type: topic
---

# Moral Dilemma Dataset (MDD) Overview

The Moral Dilemma Dataset (MDD) is a distributional, pluralism-oriented benchmark designed to evaluate the alignment between large language models (LLMs) and the spectrum of human moral judgment in authentic, context-rich scenarios. Unlike prior datasets focused on stylized, synthetic dilemmas or single-label crowd judgments, MDD combines real-world moral ambiguity, granular distributional human annotations, and rigorously engineered value taxonomy to probe both the outcomes and rationales of LLM moral reasoning.

## 1. Composition, Source, and Construction

The MDD comprises 1,618 true-to-life moral dilemmas sourced from the r/AmITheAsshole (AITA) subreddit over a six-month period, representing diverse domains such as family conflict, workplace ethics, care obligations, and interpersonal disputes. Each scenario is fully decontextualized—rewritten (by GPT-4o-mini) to remove distinguishing AITA formatting or signals—thereby capturing an ecologically valid and hard-to-spoof distribution of real dilemmas.

For each scenario, both a succinct title and a detailed narrative body are provided, along with manually assigned topic labels (family, work, health, etc.) and available demographic cues. Scenarios were selected for inclusion only if they could not be trivially identified as originating from AITA, ensuring general-purpose utility.

## 2. Human Annotation and Judgment Distribution

Each dilemma is annotated with the complete set of direct community moral judgments: 51,776 binary human evaluations (Acceptable/Unacceptable) and their accompanying free-text rationales, extracted from Reddit comments. To enforce comparability, only majority-verdict (“NTA” or “YTA”) comments with an explicit binary stance were retained; ambiguous or non-binary (e.g., ESH, NAH, INFO) replies were excluded.

Unlike prior “consensus-first” datasets, MDD preserves the entire *pluralistic* distribution of judgments for every scenario, not just the modal outcome:

| Dilemma       | # Human Votes | % Acceptable | % Unacceptable | Consensus Score |
|---------------|--------------|--------------|----------------|-----------------|
| Example A     |      90      |   0.74       |     0.26       |      0.74       |
| Example B     |      18      |   0.56       |     0.44       |      0.56       |

Consensus score (modal class proportion, 0.5 = maximal disagreement, 1.0 = unanimous) is recorded per dilemma. This allows for detailed stratification of “clean-cut” vs. “contentious” cases during model evaluation.

Each human judgment consists of:

- A binary label: 1 (Acceptable) or 0 (Unacceptable).
- A free-text rationale (mean length: 28.9 tokens, stdev: 11.7), providing the explicit moral reasoning behind the choice.

## 3. Taxonomy of Moral Values and Value Extraction

All rationales (human and model-generated) are processed with the Value Kaleidoscope system, which applies LLM-powered classification (validated by prior work) to extract explicit value terms from moral rationales. The full procedure yields:

- 3,783 unique value expressions, mapped via semantic embedding (OpenAI `text-embedding-3-large`) and agglomerative clustering.
- Manual expert curation (5 reviewers) produces a 60-value taxonomy encompassing Autonomy, Beneficence, Care, Compassion, Justice, Inclusivity, Freedom, and 53 additional fine-grained categories.
- Each rationale yields a value profile—the set of values referenced, frequency-normalized.

For entropy analysis, value diversity per response set is calculated as:

\[ H = -\sum_{k=1}^{K} p_k \log p_k \]

where \( p_k \) is the frequency of value \( v_k \) in either human or model rationales.

## 4. Benchmarking Large Language Models: Protocol

MDD is structured for benchmarking in a distributional and rationalist fashion. Each LLM is tasked to, for every dilemma:

- Generate as many binary evaluation+rationale pairs as the number of human judgments for that case (replicating sample size and diversity).
- Rationale generation is evaluated both for surface-level label agreement and for value diversity and distributional alignment.

**Prompting Regimes**:

- **Zero-shot**: LLMs respond without demographic or persona cues.
- **Persona-based**: Demographic sampling to match community diversity.
- **Model council**: Aggregated responses from multiple LLMs acting as a panel.

**Distributional Alignment Metric**:

For each dilemma \( d_i \) with \( N_i \) human judgments,

\[
\Delta_i = \left|P^{\text{Human}_i}(1) - P^{\text{LLM}_i}(1)\right|
\]

where \( P^{\text{Human}_i}(1) \) is the empirical proportion of “Acceptable” in human judgments, and \( P^{\text{LLM}_i}(1) \) is the corresponding model proportion. Mean \( \Delta \) is reported across all dilemmas; lower is better.

## 5. Major Findings on Model-Human Alignment

MDD establishes two orthogonal axes of evaluation:

1. **Distributional Judgment Alignment**: LLMs reproduce human consensus well only when it is high; in ambiguous scenarios with discordant human judgments, LLMs default to mode-seeking, resulting in poor distributional alignment even for best-in-class models (council \(\Delta = 0.22\)). Standard prompting does not capture humanlike pluralism in ambiguous cases.

2. **Value Diversity Gap**: Model rationales rely on a much narrower palette of moral values. Human rationales produce an entropy \(H_{human} = 0.57\), with the top 10 values accounting for only 35.2% of value mentions. LLMs, by contrast, have entropy \(H_{model} = 0.37\) (standard regime), top 10 covering 81.6%—focusing on Autonomy, Care, and similar platitudes, rarely reflecting marginalized or situational values prominent in human justification.

A key intervention, Dynamic Moral Profiling (DMP)—conditioning model responses on sampled human value-profiles from a Dirichlet prior—boosts both alignment (\(-64.3\%\) mean error, especially in low-consensus cases) and entropy (\(H_{DMP} = 0.52\)), yielding greater coverage of mid- and low-frequency values.

## 6. Methodological Advantages and Applications

MDD enables pluralism-sensitive, real-world grounded benchmarking for LLMs, providing:

- Distributional direct comparison metrics for models’ ability to reflect true human heterogeneity.
- Rationalist value-profile assessment, enabling analysis of which values are being invoked or neglected by LLMs in their justification.
- Ecologically valid test bed for interventions such as DMP or theory-driven prompting (e.g., Moral Foundations Theory filtering).

MDD is well-positioned for diagnosing risks of monist/majoritarian alignment, testing LLMs intended for advisory roles, and for evaluating the effectiveness of context- or value-conditioning techniques at surfacing humanlike diversity in automated ethical advice.

## 7. Comparison and Positioning within Moral Judgment Datasets

Compared to prior datasets:

| Dataset           | Scale    | Dilemma Source       | Label Distribution | Value Taxonomy | Free-text Rationales |
|-------------------|----------|----------------------|-------------------|---------------|---------------------|
| MDD (this work)   | 1.6k     | Decontextualized real world | Full, per-dilemma   | 60-values      | Yes                 |
| Scruples          | 32k+     | Reddit-AITA          | Full, multiclass   | No            | No                  |
| Moral Machine     | 40M      | Synthetic AV         | Aggregate AMCE     | None          | No                  |
| ETHICS            | 130k     | Synthetic/fictional  | Binary             | Theory-mapped | No                  |
| UniMoral          | 194–294/scenario x 6 lang | Psych + Reddit   | Per-annotator   | 4 Principle types    | Yes                 |

MDD’s design is unique for capturing both the judgment *distribution* and the *diversity of rationales* as produced by naturalistic communities, operationalizing *value pluralism* as a concrete evaluation axis [2507.17216].

## 8. Implications and Future Directions

The MDD reveals that distributional human alignment and value pluralism are not realized by default in even the strongest language models, underscoring the need for explicit, data-driven conditioning if LLMs are to play credible roles in sensitive, ethically-charged contexts. Pluralistic distributional benchmarks, such as MDD, provide a more stringent and informative test of moral alignment than single-label or stylized datasets.

The data and methodology demonstrate the inadequacy of majority-vote metrics for capturing human-moral complexity and provide a scalable, extensible foundation for future research on value alignment, model-centric pluralism interventions, and pluralistic explainability in AI moral reasoning.

---
**References:**  
- "The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models" [2507.17216]
- "Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes" [2008.09094]
- "Mapping Topics in 100,000 Real-life Moral Dilemmas" [2203.16762]

Source: https://www.emergentmind.com/topics/moral-dilemma-dataset-mdd