ValueDCG: Assessing LLM Value Alignment

• ValueDCG is a quantitative metric that evaluates LLMs' comprehension of human values by testing their ability to both identify (know what) and explain (know why) value-based outputs.
• It employs a two-stage method, measuring discrimination against reference answers and justifications using similarity-based scoring, grounded in the Schwartz Value Survey.
• Empirical findings reveal that while larger models excel in value discrimination, the quality of explanations plateaus, raising important concerns for safe LLM deployment.

ValueDCG (Value Discriminator–Critique Gap) is a quantitative metric designed to assess the depth and coherence of LLMs' (LLMs) understanding of human personal values. It jointly evaluates two complementary aspects: whether an LLM can accurately discriminate which specific value its output reflects (“know what”) and if it can produce reasoned, value-grounded explanations for that choice (“know why”). By formalizing both facets and measuring their alignment, ValueDCG enables fine-grained analysis of value understanding beyond surface-level compliance, providing essential safety signals for LLM deployment in decision-critical contexts (Zhang et al., 2023).

1. Motivation and Conceptual Foundations

LLMs increasingly inform and influence human decisions in high-stakes scenarios, making alignment with human values critical for safety and trustworthiness. Conventional benchmarks predominantly verify if an LLM “says the right thing” but rarely probe if the model genuinely understands the underlying value frameworks or the rationale justifying its outputs. ValueDCG addresses this deficit through a two-part lens:

• “Know what” assesses the model’s capacity to self-discriminate among nuanced value categories (e.g., distinguishing “Benevolence” from “Achievement”).
• “Know why” examines the model’s ability to justify its classification with coherent, substantive reasoning anchored in human values.

This dual measurement exposes scenarios where models generate plausible but superficial explanations or produce correct attributions without substantive rationales, both of which are insufficient for robust value alignment.

2. Formal Definition and Measurement Protocol

ValueDCG operationalizes the two core components—discrimination and critique—using a standardized evaluation pipeline grounded in the Schwartz Value Survey. Each LLM under test is presented with open-ended, value-probing questions for which it generates responses, self-selects the closest baseline value-oriented answer, and justifies that selection.

Let $M = \{m_1, \ldots, m_n\}$ denote the set of LLMs, and $S = \{x_1, \ldots, x_k\}$ the evaluation set, where each item $x$ comprises:

• $x^q$: a value-probing question
• $$x^{ba} = \{\text{answer}_{v_1}, \ldots, \text{answer}_{v_{10}}\}$$: one reference answer per Schwartz value
• $$x^{br} = \{\text{reason}_{v_1}, \ldots, \text{reason}_{v_{10}}\}$$: corresponding gold explanations

The protocol proceeds as follows:

1. Value Choice: The LLM $m$ generates its answer and is prompted to select the closest reference answer’s value label.
2. Discriminator (“Know What”): Compute $$Q_{\mathrm{dis}}(m, x, v_c) = F(m(x^q), \text{answer}_{v_c})$$, where $F(\cdot,\cdot)$ denotes a similarity score in $[0,1]$, operationalized by GPT-4.
3. Critique (“Know Why”): Compute $S = \{x_1, \ldots, x_k\}$0, scoring the similarity of self-explanation to the gold rationale.
4. ValueDCG: The expected absolute difference over the dataset,

$S = \{x_1, \ldots, x_k\}$1

or, more generally, as a weighted sum over items.

A small ValueDCG indicates strong concordance between value discrimination and justification. A high ValueDCG signals that one aspect is outpacing the other, exposing a comprehension gap.

3. Engineering Implementation and Evaluation Dataset

The ValueDCG framework leverages a rigorously constructed evaluation set:

• Reference Pool: Based on the Schwartz Value Survey’s ten universal value categories.
• Itemization: 100 distinct open-ended prompts, each paired with ten high-quality reference answers and explanations, resulting in 1,000 reference answer/explanation pairs.
• Labeling and Scoring: GPT-4 is tasked with two labeling roles: “Value Similarity Labeler” scores value content alignment; “Reasoning Similarity Labeler” rates coherence with gold-standard rationales. Scores are dictionary-mapped per value.
• Model Settings: LLMs are tested at temperature 0.0, top-p 0.95 for deterministic output. Contexts include “no induction” (direct prompt) and “value induction” (contextual priming).
• Verification: Human annotators validate a subset of GPT-4 similarity scores, observing high agreement rates.
• Aggregation: For each model and context, average absolute discriminator–critique differences are reported.

The evaluation pipeline is explicitly designed to stress-test self-awareness and explanatory competence in LLMs with respect to value-laden reasoning.

4. Experimental Findings

The ValueDCG metric was applied to GPT-4, GPT-3.5-turbo, Llama-2-7B-chat, Llama-2-13B-chat, and Vicuna-33B, using 100 value-probing questions across 11 context permutations (one neutral, ten value primings).

Empirical outcomes can be summarized as follows:

• Discriminator (“Know What”) exhibits strong positive correlation with model size: larger LLMs increasingly succeed in aligning their responses to the correct value categories.
• Critique (“Know Why”) remains consistently high across all tested models, improving only marginally with scale and largely saturating even at modest parameter counts.
• ValueDCG diminishes with increases in model size, driven by “know what” improvements; “know why” scores show ceiling effects.
• Growth Rate Asymmetry: Quantitative evidence demonstrates discordant scaling: as parameter counts rise (7B → 13B → 33B → GPT-3.5 → GPT-4), $S = \{x_1, \ldots, x_k\}$2 increases sharply whereas $S = \{x_1, \ldots, x_k\}$3 plateau.
• Discrepancy Implication: This suggests explanation generation is “easier” to fake—models craft contextually appropriate rationales without necessarily understanding or correctly discriminating the relevant values.

5. Risks, Implications, and Blind Spots

Several substantive risks and implications are highlighted by ValueDCG analyses:

• Plausible Hallucinations: Even small models can generate human-like, convincing rationales (high “know why”) without robust value comprehension, facilitating persuasive but potentially misleading explanations.
• Context Over Moral Grounding: LLM value self-classification is highly context-sensitive; performance is dramatically affected by in-context priming rather than persistent moral grounding.
• Safety Overconfidence: A low ValueDCG, driven by high critique scores, may conceal a lack of genuine value recognition, introducing hazards in domains demanding authentic value alignment (e.g., allocation, medical guidance).
• Value Blind Spots: LLMs are less adept at recognizing or justifying items linked to “risky” values (e.g., Power, Hedonism). Safety fine-tuning may suppress output related to these domains without fostering true value understanding, presenting adversarial vulnerabilities.

6. Limitations and Prospective Research Directions

Several limitations and opportunities for refinement are identified:

• Similarity Labeling Limits: Current reliance on GPT-4 for scoring may lack nuanced granularity for subtle value distinctions. Advancements may require dedicated value-similarity encoders.
• Value Set Breadth: The focus on Schwartz’s ten values omits broader cultural, philosophical, and situational nuances of human morality. Broader coverage and culturally adaptive frameworks are warranted.
• Module Integration: The present approach isolates discrimination and critique; future work may pursue integrated or adversarially trained modules (e.g., self-debate) to probe more robust value understanding.
• Interactional Dynamics: Extending ValueDCG to multi-turn settings and real-world deployment logs could reveal how value understanding evolves in ongoing interaction.

ValueDCG offers a robust formalism for empirically probing the alignment and authenticity of LLMs’ engagement with human values, uncovering both the strengths and structural limitations of current-generation models in value understanding and justification (Zhang et al., 2023).