Latent Value Hypothesis in AI Alignment

• The latent value hypothesis is a theory that pretraining on large-scale data encodes human values as latent directions in a model's internal representation.
• It explains how constitutional prompts can activate these latent values, shifting generation policies to align AI behavior without new external labels.
• This framework unifies insights from reinforcement learning and cultural steering, detailing both the improvements and potential pitfalls in AI alignment.

Searching arXiv for the primary paper and closely related latent-value papers to ground the article in current literature. The latent value hypothesis is an account of why Reinforcement Learning from AI Feedback (RLAIF) can improve alignment without new external labels: pretraining on internet-scale data encodes human values as latent directions in a model’s internal representation space, a constitution prompt elicits those directions into explicit preference judgments, and preference optimization then “wires up” the elicited directions to the generation policy (Young, 3 Mar 2026). In this formulation, values are not created by alignment training; they are already present in representation space and become behaviorally salient when the prompt or training procedure activates them. Subsequent work has operationalized the same general idea in cultural evaluation and activation steering, where latent value structure is probed through scenario-based dilemmas and manipulated at inference time (Dang et al., 25 May 2026).

1. Definition and conceptual scope

In its precise alignment-theoretic form, the latent value hypothesis states that pretraining encodes human values as latent directions in representation space, while constitutional prompts act as elicitation mechanisms that selectively surface those latent value directions into preference judgments (Young, 3 Mar 2026). RLAIF then improves alignment by shifting the generation policy toward the elicited judgment direction. The hypothesis is therefore representational rather than purely behavioral: the central claim is about the geometry of hidden states, not merely about observed outputs.

This framing distinguishes between at least three objects. First, there is a representation function

$$h:(x,y)\in \mathcal{X}\times \mathcal{Y}\mapsto h(x,y)\in \mathbb{R}^d,$$

assumed whitened and centered so that $\mathbb{E}[h]=0$ and $\mathbb{E}[hh^\top]=I$ (Young, 3 Mar 2026). Second, there is a latent value direction $v^*$ encoding “true safety” or “true harm.” Third, there are prompt- or training-dependent directions that determine what the model generates and what it judges. The basic explanatory move is that these latter directions need not coincide.

A broader, culturally oriented formulation appears in work on cultural value alignment, where latent cultural values are defined as internal, pre-trained value priors encoded in hidden states and mapped onto the Inglehart–Welzel axes of Traditional vs. Secular-Rational and Survival vs. Self-Expression (Dang et al., 25 May 2026). That work does not replace the alignment-theoretic formulation; rather, it operationalizes the claim that latent values can remain inaccessible to direct prompting yet recoverable through more discriminating probes.

A conceptually related but distinct use of latent value appears in reinforcement learning, where value is decomposed into a predicted latent future and a policy-independent return evaluator (Tang et al., 2021). This is not the same hypothesis as the alignment account of constitutions and preference judgments. It nevertheless suggests a broader research motif: value can be treated as a structure emergent from latent representations rather than as a quantity learned only from explicit scalar supervision.

2. Representational formalization

The core formal assumption is linear value encoding:

$S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$

where $\epsilon$ is mean-zero noise independent of $h$ with $\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$ (Young, 3 Mar 2026). In this model, scalar safety is the linear projection of the representation onto $v^*$. The corresponding encoding quality is

$$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$

which measures how well values are linearly encoded in $\mathbb{E}[h]=0$0.

Generation and judgment are represented separately. The base model’s generation policy is assumed to optimize a linear energy,

$\mathbb{E}[h]=0$1

where $\mathbb{E}[h]=0$2 is the generation direction (Young, 3 Mar 2026). Constitutional preference judgments are modeled with a Bradley–Terry form,

$\mathbb{E}[h]=0$3

where the constitution $\mathbb{E}[h]=0$4 activates a judgment direction $\mathbb{E}[h]=0$5 (Young, 3 Mar 2026). When the constitution is well designed, $\mathbb{E}[h]=0$6 ideally aligns with $\mathbb{E}[h]=0$7; when it is not, it may activate a proxy or a harmful direction.

The same framework admits a subspace interpretation. One may define a value-relevant subspace $\mathbb{E}[h]=0$8 spanned by value directions, with $\mathbb{E}[h]=0$9 denoting orthogonal projection onto that subspace (Young, 3 Mar 2026). Conceptually, a constitution then acts like a selection or projection mechanism into $\mathbb{E}[hh^\top]=I$0, eliciting value-relevant components of $\mathbb{E}[hh^\top]=I$1 by activating some $\mathbb{E}[hh^\top]=I$2. The formal results do not require an explicit projector, but the subspace view helps connect the hypothesis to low-rank safety structure.

Correlation is measured by cosine similarity,

$\mathbb{E}[hh^\top]=I$3

Under isotropy, inner products are proportional to correlations (Young, 3 Mar 2026). This makes the geometry of value alignment explicit: whether RLAIF helps depends on how the judgment direction and the generation direction each relate to the latent value direction.

3. Mechanism in RLAIF

Under a constitutional reward

$\mathbb{E}[hh^\top]=I$4

and a KL penalty $\mathbb{E}[hh^\top]=I$5, Direct Preference Optimization yields

$\mathbb{E}[hh^\top]=I$6

so RLAIF shifts the generation direction from $\mathbb{E}[hh^\top]=I$7 to $\mathbb{E}[hh^\top]=I$8 (Young, 3 Mar 2026). In this account, alignment training does not discover a new value representation; it moves generation toward a direction already latent in the model’s representation space.

Alignment is defined as

$\mathbb{E}[hh^\top]=I$9

For small $v^*$0,

$v^*$1

where $v^*$2 is positive definite (Young, 3 Mar 2026). Improvement occurs if and only if $v^*$3. Under isotropy, this simplifies to $v^*$4, equivalently $v^*$5.

A stronger comparative statement explains why self-improvement is possible. If $v^*$6, then for small $v^*$7, moving toward $v^*$8 increases alignment relative to the base (Young, 3 Mar 2026). This is the formal statement of the generation–judgment gap: the model may judge using a direction better aligned with latent values than the one it ordinarily uses to generate text.

The paper makes that gap explicit by assuming that only a small fraction $v^*$9 of pretraining is value-relevant, while $S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$0 is value-neutral. In that regime,

$S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$1

so

$S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$2

(Young, 3 Mar 2026). The claim is not that the base model lacks value information; it is that generation is diluted by the predominance of value-neutral predictive structure.

The same framework also supplies a failure mode. If a constitution activates a direction with negative alignment to $S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$3, then RLAIF degrades alignment. The toy example with $S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$4, $S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$5, and an adversarial $S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$6 illustrates exactly this possibility (Young, 3 Mar 2026).

4. Ceiling effects, adversarial constitutions, and empirical unification

The latent value hypothesis does not claim that constitutional elicitation can recover arbitrary value quality. With $S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$7, the best attainable RLAIF policy is bounded by the encoding quality:

$S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$8

for any constitution $S(x,y)=\langle h(x,y),v^*\rangle+\epsilon(x,y),$9 (Young, 3 Mar 2026). Even when $\epsilon$0, judgments rely on the proxy $\epsilon$1 rather than on the full scalar safety signal $\epsilon$2, so unencoded or noisy value information produces irreducible regret.

This immediately yields a scaling claim: if $\epsilon$3 increases with model capacity and pretraining diversity, then the RLAIF ceiling increases with scale (Young, 3 Mar 2026). The paper states that this is empirically consistent with labeler-size scaling in RLAIF, where larger models produce better constitutional judgments and hence better alignment. A plausible implication is that increasing preference data volume alone cannot remove a ceiling imposed by inadequate value encoding.

The same formalism predicts adversarial constitutions. If pretraining encodes both a pro-social $\epsilon$4 with $\epsilon$5 and an anti-social $\epsilon$6 with $\epsilon$7, then there exists a constitution $\epsilon$8 whose activated direction satisfies $\epsilon$9, implying

$h$0

(Young, 3 Mar 2026). An adversarial constitution is therefore any constitution for which $h$1.

The framework is presented as a unification of several empirical findings. First, a single refusal direction is said to exist already in base models; under the hypothesis, alignment training modifies $h$2 but does not create $h$3 (Young, 3 Mar 2026). Second, low-rank safety structure is accommodated by the value concentration conjecture: if $h$4 is the pre-whitened representation covariance with eigenpairs $h$5, then

$h$6

where

$h$7

(Young, 3 Mar 2026). This is consistent with empirical reports of rank-1 to low-rank safety directions.

The cultural steering literature supplies an independent operational corroboration. Direct WVS-style prompting triggered refusals for 10% of items in Llama 3.2–3B under basic prompting, rising to 30% with advanced persona prompts, while Qwen3–4B answered all queries but 40% of its responses were locked at mid-scale safety scores of 5–6/10 (Dang et al., 25 May 2026). Scenario-based probing with implicit token probabilities bypassed these surface behaviors, suggesting that latent value structure may be present even when direct elicitation fails.

5. Operationalization through probing and steering

The cultural-alignment framework provides a concrete methodology for recovering and intervening on latent value directions without retraining (Dang et al., 25 May 2026). It constructs 600 situational dilemmas, 200 per domain, with experiments split into 300 items for steering-vector optimization and 300 for evaluation. Each dilemma forces a binary choice between options aligned to opposite poles of a WVS-derived axis, with randomized A/B assignment.

Given logits $h$8 and $h$9 for the final decision token, the probability of choosing the positive axis is

$\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$0

Axis coordinates are then estimated by averaging probabilities:

$\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$1

followed by the rescaling

$\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$2

(Dang et al., 25 May 2026). This makes latent value coordinates observable even when overt self-reports are neutralized by refusal behavior.

Steering vectors are extracted by contrastive mean differences:

$\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$3

and applied during inference through

$\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$4

(Dang et al., 25 May 2026). The reported layer sets are Qwen layers 16–19, Llama layers 8, 9, 11, 12, and Gemma layers 12–15; $\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$5 maintained baseline perplexity and coherence for Qwen and Gemma, while Llama required $\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$6 to overcome rigidity.

The most salient result is latent entanglement. Steering along one cultural axis consistently shifted the other, producing diagonal rather than axis-aligned motion on the cultural map. The entanglement ratio

$\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$7

ranged from 0.7 to 0.9, while the corresponding WVS axes in human data correlate at $\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$8 (Dang et al., 25 May 2026). The authors interpret this as evidence that models encode a coupled “Modernity” feature rather than two orthogonal dimensions.

This does not refute the latent value hypothesis. It suggests, rather, that latent values may inhabit structured and coupled manifolds rather than a set of neatly separable one-dimensional axes. A plausible implication is that constitutions and steering interventions may routinely activate mixtures of latent features, which helps explain trade-offs and side effects in alignment interventions.

A distinct but related decomposition appears in value estimation for reinforcement learning. There, the expected return is written as a composition of a predictive latent dynamics module

$\mathrm{Var}(\epsilon)=\sigma_\epsilon^2$9

and a policy-independent return functional

$v^*$0

(Tang et al., 2021). With linear $v^*$1, the decomposition is exact; with convex $v^*$2, it yields a lower bound by Jensen’s inequality. Although this work addresses reinforcement learning rather than constitutional alignment, it supports the more general view that value can be mediated by latent structure.

6. Practical implications, limitations, and open problems

The practical recommendations that follow from the latent value hypothesis are concrete. Constitutions should directly query harm, harmlessness, honesty, and related properties so that the activated direction $v^*$3 is more likely to align with $v^*$4 (Young, 3 Mar 2026). Validation should measure $v^*$5 using linear probes for toxicity or refusal, and constitutions may be ensembled or mixed to reduce sensitivity to any single mis-specified direction. The same work recommends empirically testing downstream effects rather than relying on constitution surface semantics alone, treating constitution design as an attack surface, and monitoring changes across training iterations.

The hypothesis also changes how alignment resources are allocated. The paper explicitly recommends prioritizing labeler capacity and pretraining diversity, which raise $v^*$6, over sheer preference data volume (Young, 3 Mar 2026). It further recommends using RLHF to cover values not well encoded in pretraining, positioning RLAIF as a method for eliciting already encoded values rather than generating genuinely novel normative content.

Several limitations are explicit. The linear encoding assumption is a simplification; values may be nonlinear, context-dependent, or non-monotone. The nonlinear generalization replaces $v^*$7 with an average gradient direction $v^*$8 and yields

$v^*$9

or under Gaussian $$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$0,

$$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$1

(Young, 3 Mar 2026). The constitution-to-direction map $$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$2 is also not modeled mechanistically, and the promptable set $$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$3 limits achievable correlation with true values even when $$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$4 is high.

The single-direction formulation of “true harm” is another deliberate simplification. In the multi-objective extension, if multiple value directions $$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$5 exist, then first-order improvement on each objective is

$$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$6

and Pareto improvement requires $$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$7 to lie in the cone

$$\rho^2=\frac{\|v^*\|^2}{\|v^*\|^2+\sigma_\epsilon^2} =\frac{\mathrm{Var}(\langle h,v^*\rangle)}{\mathrm{Var}(S)}\in[0,1],$$8

(Young, 3 Mar 2026). This makes normative trade-offs explicit rather than treating them as residual implementation detail.

The cultural steering results add further constraints. Entanglement prevents orthogonal control; country-level proxies gloss over subcultural variation; linear steering may not capture non-linear value interactions; and the impact on general capabilities remains to be assessed (Dang et al., 25 May 2026). These findings suggest that even when latent value directions are accessible, precise alignment may be limited by the geometry of the representation itself.

Taken together, the literature defines the latent value hypothesis as a representational theory of alignment: values are encoded during pretraining, elicited by prompts or probes, and behaviorally expressed when optimization or steering shifts the effective generation direction toward those encoded structures (Young, 3 Mar 2026). Its explanatory power lies in accounting for the generation–judgment gap, the ceiling imposed by encoding quality, the existence of adversarial constitutions, and the empirical appearance of refusal directions and low-rank safety subspaces. Its main unresolved questions concern the mechanistic map from prompt to activated direction, the degree of nonlinear and multi-objective coupling in value space, and robust defenses against constitutions or interventions that activate harmful latent directions.