---
title: DVGR in LLM Value Generalization
url: https://www.emergentmind.com/topics/deep-value-generalization-rate-dvgr
type: topic
---

# DVGR in LLM Value Generalization

Searching arXiv for the benchmark paper and the earlier predictive-values paper to ground the article and verify the terminology.
Deep Value Generalization Rate (DVGR) is the central metric in the Deep Value Benchmark (DVB) for quantifying whether large language models (LLMs) generalize users’ underlying moral values rather than superficial, stylistic, or otherwise shallow preferences when learning from preference data [2511.02109]. In DVB, DVGR is defined within a confound-then-deconfound in-context learning paradigm: values and shallow features are deliberately correlated during training demonstrations and then decoupled at test time, so that the measured quantity is the expected proportion of test cases where a model predicts the option aligned with the deep value, not the shallow feature [2511.02109]. A separate paper uses the same acronym for a different construct derived from generalization bounds for predictive values of thresholded scoring functions, defining a “Deep Value Generalization Rate” as the leading-order decay rate of PPV/NPV estimation error with sample size \(n\) [2007.05073]. This terminological overlap suggests that, in current usage, DVGR is primarily a benchmark metric in alignment research, while also appearing as a theoretically motivated shorthand in predictive-value learning theory.

## 1. Definition and core intuition

In the DVB formulation, DVGR measures whether a model projects the underlying deep value \(D\) guiding a user’s choices—rather than the shallow feature \(S\) that happened to co-occur with \(D\) during training—to a new situation where \(D\) and \(S\) are decoupled [2511.02109]. A DVGR of \(1\) indicates perfect deep-value generalization across decoupled test instances; a DVGR of \(0\) indicates perfect shallow-preference generalization; chance for two-alternative choices is \(0.5\) [2511.02109]. The metric is therefore designed to distinguish value generalization from mere pattern matching over correlated surface cues.

The formal setup uses a deep value dimension \(D \in \{d_1,d_2\}\) and a shallow feature dimension \(S \in \{s_1,s_2\}\) for an experimental tuple \(c = (d_1,d_2,s_1,s_2,\text{context})\) [2511.02109]. Training demonstrations pair \((d_1,s_1)\) against \((d_2,s_2)\) and consistently label \((d_1,s_1)\) as preferred; test instances then swap shallow features and present \((d_1,s_2)\) versus \((d_2,s_1)\) [2511.02109]. Under this construction, a deep-value generalizer should select \((d_1,s_2)\), whereas a shallow-feature generalizer should select \((d_2,s_1)\).

For a test set \(T\) of size \(N\), the indicator of value alignment is
\[
Y_i = 1 \quad \text{if the model predicts the option embodying } (d_1,s_2),
\]
and
\[
Y_i = 0 \quad \text{if the model predicts the option embodying } (d_2,s_1).
\]
DVGR is then
\[
\text{DVGR} = \frac{1}{N}\sum_{i=1}^{N}\mathbb{I}[\text{model chooses option aligned with deep value}],
\]
or equivalently
\[
\text{DVGR} = \mathbb{E}_{i\sim T}[Y_i].
\]
The same paper also gives the test-time form
\[
\text{DVGR} = \frac{1}{N}\sum_{i=1}^{N}\mathbb{I}[\hat{y}_i = (d_1,s_2)].
\]
Less-than-chance performance is assessed with a binomial test against \(p_0=0.5\), and confidence intervals use Wilson or Clopper–Pearson intervals [2511.02109].

## 2. Confound-then-deconfound experimental schema

The DVB design is built around a controlled confounding scheme in training and a decoupled testing distribution [2511.02109]. In training, the user consistently prefers \((d_1,s_1)\) over \((d_2,s_2)\), so the training distribution makes the deep value \(d_1\) and the shallow feature \(s_1\) equally predictive of the preference label. Formally, for the training choice set \(X_{\text{train}} = \{((d_1,s_1),(d_2,s_2))\}\), the training labels indicate
\[
\Pr(\text{choose }(d_1,s_1)\mid \text{train}) = 1.
\]

At test time, shallow features are swapped, yielding \(X_{\text{test}} = \{((d_1,s_2),(d_2,s_1))\}\) [2511.02109]. The model is then evaluated without labels, and the metric records the proportion of choices aligned with \(d_1\). This creates a clean conflict between two hypotheses: one based on the underlying value and one based on the surface feature. The benchmark’s interpretability follows from the fact that it isolates which signal a model learned from preference data—deep moral value versus shallow surface correlation—by creating a controlled conflict between equally predictive training features and then decoupling them at test [2511.02109].

The canonical illustrative schema is: training prefers \((\text{non-maleficence}, \text{formal})\) over \((\text{justice}, \text{informal})\), while test presents \((\text{justice}, \text{formal})\) versus \((\text{non-maleficence}, \text{informal})\) [2511.02109]. In this setting, a model that relies on “formal” will choose justice+formal, whereas one that relies on the deep value will choose non-maleficence+informal. This design makes DVGR a direct measure of generalization under distribution shift induced by unconfounding.

## 3. Benchmark construction: values, shallow features, and contexts

The deep-value inventory in DVB combines prima facie duties from Ross and Schwartz social values [2511.02109]. The prima facie duties are beneficence, fidelity, justice, non-maleficence, reparation, and self-improvement, with gratitude excluded from the final benchmark after pilot [2511.02109]. The Schwartz values are security, conformity, tradition, universalism, and benevolence [2511.02109].

Shallow preferences were generated via GPT-4o candidates and then filtered by human validation for shallowness, neutrality, defined as balanced poles, and breadth, defined as domain applicability [2511.02109]. The top 20 dichotomies were selected; examples include formality vs informality, frequent vs minimal feedback, and adaptive vs static behavior [2511.02109]. This filtering is important because the benchmark depends on the shallow feature being separable from the deep value while still being plausible in realistic preference data.

Contexts were grounded in Y Combinator’s “AI Assistant” startups and O*NET work activities [2511.02109]. The benchmark uses eight domain clusters: commerce, customer service, finance, productivity, communication, healthcare, legal, and education [2511.02109]. For each domain, the top 10 O*NET activities were chosen via standardized relevance, specifically z-scored importance and level aggregated to cluster level [2511.02109].

The universe \(U\) is a factorial combination of deep value pairs, shallow preference pairs, and contexts, filtered to a sample \(S\) consisting of \(50\) \((v_1,v_2,s_1,s_2)\) pairings per context across \(8\) contexts, yielding \(400\) tuples \(c\) [2511.02109]. For each tuple, the benchmark generates \(40\) training scenarios with consistent preference for \((v_1,s_1)\) over \((v_2,s_2)\) and \(40\) swapped-pairing test scenarios [2511.02109]. For each \(c\), the protocol presents \(N \in \{5,20,40\}\) in-context training examples followed by \(10\) decoupled test questions, for a total of \(12{,}000\) administered test questions [2511.02109]. Responses are extracted in isolation, with one test per prompt, to avoid context pollution [2511.02109].

## 4. Estimation, prompting, and statistical analysis

The prompting setup instructs models to answer “Option A” or “Option B” only, with max tokens \(=10\), default temperature, and one test per prompt [2511.02109]. Whether \((v_1,s_1)\) appears as A or B is randomized in generation to control positional bias, and the order of \((v_1,s_2)\) and \((v_2,s_1)\) is randomized at test as well [2511.02109]. Extraction failures, meaning non-conforming outputs, are treated as missing data; there are no ties or abstentions by design [2511.02109].

Scoring maps Option A or B to metadata indicating which option is \((v_1,s_2)\) versus \((v_2,s_1)\), assigns \(1\) if the model picked \((v_1,s_2)\) and \(0\) otherwise, and defines DVGR as the mean score across valid trials [2511.02109]. The benchmark also reports an adjusted DVGR from a mixed-effects logistic regression per model to adjust for value-level predispositions:
\[
\text{logit}(\Pr(Y=1)) = \beta_0 + \alpha_{[v_1]},
\]
with random intercepts \(\alpha_{[v_1]}\), and adjusted DVGR \(=\text{logistic}(\beta_0)\) [2511.02109]. Raw and adjusted DVGR were found to be near-identical, with mean absolute difference \(0.003\) [2511.02109].

The main inferential machinery comprises Wilson confidence intervals for proportions, Clopper–Pearson as an alternative, binomial tests against chance \(p_0=0.5\), \(\chi^2\) tests with Cramer’s \(V\) effect sizes for group comparisons, and logistic regression with clustered standard errors for multivariate factors [2511.02109]. In the reported implementation, models answered \(97\%\) of trials, and the analysis dataset contains \(N=104{,}725\) test decisions across nine models [2511.02109].

| Component | DVB specification |
|---|---|
| Output format | “Option A” or “Option B” only |
| Prompt structure | One test per prompt |
| Training-shot counts | \(N \in \{5,20,40\}\) |
| Valid-trial scoring | \(1\) for \((v_1,s_2)\), else \(0\) |
| Chance baseline | \(0.5\) |
| Main CI | Wilson |
| Main significance test | Binomial against \(0.5\) |

This estimation procedure makes DVGR an operational proportion rather than a latent construct inferred indirectly from aggregate utility, questionnaire scores, or free-form rationales.

## 5. Empirical results in the Deep Value Benchmark

Across nine LLMs, the average DVGR is approximately \(0.30\), and all models generalized deep values less than chance, with binomial \(p<0.001\) for each model [2511.02109]. The reported raw per-model estimates, all below \(0.5\), are: gpt-4.1-mini \(0.23\), meta-llama-3-70b-instruct \(0.24\), gpt-4.1 \(0.24\), gpt-4o \(0.25\), gpt-4o-mini \(0.27\), gemini-2.0-flash-lite \(0.34\), gpt-4.1-nano \(0.35\), meta-llama-3-8b-instruct \(0.37\), and gemini-2.0-flash \(0.40\), each with Wilson intervals entirely below chance [2511.02109].

A model-size analysis found that paired comparisons of families, comparing small vs large models, show smaller models often have slightly higher DVGR in \(3/5\) pairs, with small mean absolute differences of about \(0.07\), though statistically significant given the sample size; an omnibus \(\chi^2\) test also favors smaller models [2511.02109]. The paper states that scale does not solve value generalization and that DVGR is not emergent with size, echoing inverse-scaling results in other alignment-relevant areas such as truthfulness and sycophancy [2511.02109]. This suggests that the relevant inductive bias is not simply improved by scaling the same preference-learning paradigm.

Factor analyses show heterogeneous but bounded variation across benchmark dimensions [2511.02109]. Context effects are small, with Cramer’s \(V=0.09\): higher DVGR appears in commerce, healthcare, and finance, and lower DVGR in communication, education, and customer service [2511.02109]. The number of in-context examples has negligible effect, with Cramer’s \(V=0.01\), and DVGR remains approximately \(0.30\) regardless of whether \(N=5,20,40\) demonstrations are used [2511.02109]. Value identity matters somewhat more, with Cramer’s \(V=0.18\): tradition reaches approximately \(0.51\), universalism approximately \(0.42\), and lower DVGRs are reported for fidelity and self-improvement [2511.02109].

Agreement analyses further indicate that models tend to make similar choices. Pairwise agreement is approximately \(74\%\) overall, with within-developer pairs agreeing more, approximately \(76.8\%\), than cross-developer pairs, approximately \(72.2\%\), a difference of about \(3.6\) percentage points with \(p=0.04\) [2511.02109]. A plausible implication is that low DVGR is not idiosyncratic to a single model family but reflects a shared response tendency under the benchmark’s controlled confounds.

## 6. Validation, interpretability, and alignment significance

The benchmark includes three human validation experiments intended to establish construct validity, internal validity, and external validity [2511.02109]. In the shallow-versus-deep distinction study, \(41\) Prolific participants produced shallowness ratings that robustly separate deep values from shallow preferences, with means \(-0.98\) versus \(0.34\) on the \(-2\) to \(+2\) scale, Cohen’s \(d \approx 1.05\), and mixed-model \(\beta \approx 0.92\), \(p<.001\) [2511.02109]. Binary accuracy on selected shallow preferences is approximately \(0.91\) [2511.02109].

Completion Validation 1 used \(20\) participants over \(200\) trials and found that participants predicted the user would choose the value-aligned option in approximately \(91\%\) of cases, with \(95\%\) CI \([0.86,0.94]\) and \(p \approx 2.6\times 10^{-35}\) against chance [2511.02109]. Completion Validation 2 used \(21\) participants over \(210\) trials and found approximately \(98\%\) accuracy in mapping options to intended \((\text{value},\text{preference})\) pairs, with \(95\%\) CI \([0.95,0.99]\) and \(p \approx 4\times 10^{-54}\) [2511.02109]. LLMs also succeed when explicitly told which values and preferences each option embodies, with AI performance of approximately \(0.953\) in Validation 1 and approximately \(0.987\) in Validation 2 [2511.02109]. The paper interprets this as suggesting that the difficult component for LLMs is inferring the underlying value from preference patterns when not told [2511.02109].

The benchmark’s alignment significance derives from its controlled causal structure: it directly probes whether a model learns deep moral value or shallow correlates from preference data [2511.02109]. A low DVGR indicates a tendency to generalize superficial cues such as style or formal tone rather than the underlying moral principles in similar future contexts [2511.02109]. The paper argues that this poses risks for AI assistants and agents that must act robustly under distribution shift in user contexts and values [2511.02109]. Because DVGR complements other alignment benchmarks by directly probing value-versus-style generalization under controlled confounds rather than average-case performance or static value questionnaires, it occupies a distinct niche within alignment evaluation [2511.02109].

Follow-up prompting experiments reinforce this interpretation. Chain-of-Thought lowers pooled DVGR to \(0.25\) from a baseline of \(0.30\), whereas explicit instruction to prioritize deep values raises it modestly to \(0.33\), but still below chance [2511.02109]. Examples include gpt-4.1 moving from \(0.24\) at baseline to \(0.19\) with CoT and \(0.30\) with explicit instruction, and gemini-2.0-flash moving from \(0.40\) to \(0.44\) under explicit instruction [2511.02109]. This suggests that prompting alone does not resolve the underlying inference failure.

## 7. Limitations, terminological ambiguity, and future directions

The DVB paper identifies several threats to validity and scope conditions [2511.02109]. The perfect confound in training is a “worst-case” design: real-world correlations may be partial rather than perfect, even though the perfect-confound setup enables a clean measure [2511.02109]. Deep values do not always fully determine choices, and the paper notes that differences across models, values, and contexts may be more informative than absolute levels [2511.02109]. The assessment is inference-only, using in-context learning rather than finetuning, so it may miss capabilities unlockable by post-training; the evaluation instead targets off-the-shelf models as commonly deployed [2511.02109]. Domain coverage is limited to the selected values, preferences, and eight domain clusters, and shallow features may inadvertently encode value content despite filtering for neutrality and breadth [2511.02109].

The paper reports several mitigations: human validations confirm embodiment and separability; the factorial design balances values appearing as preferred and dispreferred across tuples; and randomization of option order together with one-test-per-prompt reduces positional and contamination biases [2511.02109]. These do not remove all threats, but they strengthen the claim that low DVGR is not merely an artifact of prompt format or label leakage.

Future directions proposed in the benchmark paper include debiased or counterfactual preference datasets where value and style are explicitly decorrelated, value-focused objectives or auxiliary losses that penalize reliance on shallow features under synthetic decoupling, causal disentanglement methods to separate value representations from style features within LLM internals, and post-training finetuning specifically to increase DVGR and measure downstream behavioral benefits [2511.02109]. Evaluation extensions include more deep values, more shallow features, multi-turn interaction contexts, cross-lingual tests, domain-specific verticals such as healthcare subdomains, and interpretability analyses such as linear probes to identify representations mediating value alignment versus shallow style reliance [2511.02109].

A distinct issue is terminological ambiguity. In "Predictive Value Generalization Bounds" [2007.05073], the acronym DVGR is used for the leading-order rate controlling PPV/NPV estimation error under distribution-free uniform convergence bounds. There, for a function class \(\mathcal{F}\) and operating point \(\alpha\), the quantities
\[
\mathrm{DVGR}_{\mathrm{PPV}}(n,\mathcal{F},\alpha,\delta)
\quad\text{and}\quad
\mathrm{DVGR}_{\mathrm{NPV}}(n,\mathcal{F},\alpha,\delta)
\]
are defined through bounds depending on \(n\), \(\alpha\), \(\delta\), and either the order coefficient \(\Theta(\mathcal{F},n,k)\) or the VC-subgraph dimension \(d\) [2007.05073]. This is conceptually unrelated to DVB’s behavioral metric, despite sharing the acronym. A plausible implication is that citations should specify whether “DVGR” refers to a benchmark score for value-versus-style generalization in LLMs [2511.02109] or to a theoretical generalization-rate quantity for predictive values of thresholded scoring functions [2007.05073].

In its benchmark sense, DVGR provides a clear quantitative lens on whether models trained on preference signals learn to generalize underlying deep values or merely correlated shallow features [2511.02109]. In the reported DVB experiments, contemporary LLMs generalize deep values less than chance on average, with only modest improvements under explicit instruction [2511.02109]. Within alignment research, this makes DVGR a targeted measure of robustness to confounded preference learning rather than a broad assessment of moral competence or normative correctness.

Source: https://www.emergentmind.com/topics/deep-value-generalization-rate-dvgr