Context-Value-Action Architecture for Value-Driven Large Language Model Agents

Abstract: LLMs have shown promise in simulating human behavior, yet existing agents often exhibit behavioral rigidity, a flaw frequently masked by the self-referential bias of current "LLM-as-a-judge" evaluations. By evaluating against empirical ground truth, we reveal a counter-intuitive phenomenon: increasing the intensity of prompt-driven reasoning does not enhance fidelity but rather exacerbates value polarization, collapsing population diversity. To address this, we propose the Context-Value-Action (CVA) architecture, grounded in the Stimulus-Organism-Response (S-O-R) model and Schwartz's Theory of Basic Human Values. Unlike methods relying on self-verification, CVA decouples action generation from cognitive reasoning via a novel Value Verifier trained on authentic human data to explicitly model dynamic value activation. Experiments on CVABench, which comprises over 1.1 million real-world interaction traces, demonstrate that CVA significantly outperforms baselines. Our approach effectively mitigates polarization while offering superior behavioral fidelity and interpretability.

• The paper introduces the CVA architecture that decouples cognitive reasoning from action generation using a two-stage, generate-then-verify pipeline.
• It leverages Value-Action Mapping Calibration with SFT and DPO alongside a value-driven verifier to mitigate LLM biases and polarization.
• Empirical results on CVABench demonstrate enhanced fidelity and sustained human-like variance in behavior compared to traditional prompt-driven methods.

Context-Value-Action (CVA): A Psychologically-Grounded Framework for Value-Driven LLM Agents

Introduction

The "Context-Value-Action Architecture for Value-Driven LLM Agents" (2604.05939) addresses fundamental deficiencies in current LLM-based human-like agent architectures—specifically, their tendency toward behavioral rigidity, value polarization, and the poor emulation of realistic human variance in group-level simulations. By situating the modeling of agent behavior within the Stimulus-Organism-Response (S-O-R) psychological paradigm and operationalizing Schwartz's Theory of Basic Human Values as dynamic value activations, the paper introduces the CVA architecture. This design decouples action generation and cognitive reasoning through a value-driven verifier and leverages large-scale supervised training and preference optimization to correct LLM bias, achieving demonstrable improvements in both fidelity and interpretability.

Figure 1: Overview of the CVA framework—the agent reconstructs dynamic value activation from historical context, using these activations to verify and select actions that align with the agent’s current psychological state.

Limitations of Prevailing Agent Paradigms

Existing LLM agent paradigms—prompt-driven role-play, prompt-driven chain-of-thought (CoT), and training-required models—are fundamentally limited in their ability to represent the genuine diversity and stochasticity of human behavior. Prompt-driven approaches, especially those utilizing psychological reasoning steps, introduce cognitive distortion: instead of accurately modeling value vectors, LLMs overfit to archetypal behaviors, resulting in "caricatured" outputs aligned poorly with empirical human group distributions. This is particularly problematic given that "LLM-as-a-judge" evaluations are compromised by self-referential bias.

Figure 2: Model structure of the value verifier and the modular reasoning pipeline in the CVA architecture.

Through systematic empirical benchmarking on CVABench (comprising over 1.1 million human behavior traces), it is shown that higher inference intensity in prompt reasoning methods does not yield improved behavioral fidelity. Instead, it accelerates value polarization and rapidly erodes population variance.

The CVA Architecture

CVA adopts a two-stage, generate-then-verify pipeline to resolve the deficiencies of prompt-driven reasoning and standard black-box RL fine-tuning:

• Value-Action Mapping Calibration (VMC): The base LLM, initialized with large-scale supervised fine-tuning (SFT) and direct preference optimization (DPO), is explicitly trained on context, value, and action triplets to reconstruct authentic human-like context-conditioned value-action mappings. DPO is leveraged to amplify nuanced, contextually sensitive action preferences while suppressing degenerate, polarized response modes.
• Value-Driven Verifier Reasoning (VDR): A standalone verifier, trained discriminatively on aligned human data, evaluates the value consistency of candidate actions generated by the calibrated model. The final action is selected based on maximized alignment with the dynamically activated value profile, disentangling the underlying cognitive drivers from surface-level generation artifacts.

This modular decoupling enables transparent attribution of decision drivers and explicitly models the psychological process by which context triggers value activations that, in turn, mediate action selection.

Empirical Evaluation: Benchmarking Fidelity and Diversity

Baseline Rigidity and Polarization

Figure 3 illustrates the central pathology afflicting prompt-driven agents: increased reasoning depth consistently reduces variance and pushes behavioral distributions toward the extremal bounds of the value space, sharply contrasting with the diversity and moderate central tendencies of human baselines.

Figure 3: Polarization and solidification in the Stimulation dimension—higher reasoning intensity collapses group value diversity.

Ablation analysis (Figure 4) demonstrates that only the integration of all CVA components (SFT, DPO, and verifier) sustains high behavioral accuracy and lexical fidelity while preventing the collapse of population-level diversity.

Figure 4: Sentiment accuracy trends with increased inference intensity; decision quality saturates after modest depth, with diminishing returns and rising rigidity beyond this point.

Comparative benchmarking on CVABench further corroborates these findings: role-play and prompt-driven reasoning agents, even when powered by advanced LLMs, either fail to approach human-like intra-individual fidelity or produce highly rigid, polarized population-level distributions.

Distributional and Structural Interpretability

The latent embeddings produced by the value-guided verifier reconstruct the theoretical circular topology of Schwartz's value system (Figure 5), validating that the model internalizes the relational semantics of the basic value dimensions.

Figure 5: Learned verifier embeddings, preserving the circular structure of Schwartz’s value system, with deviations only at specific value transitions.

Moreover, the semantic landscapes extracted from the verifier’s cross-attention mechanisms (Figure 6) expose the lexical triggers for value activation, supporting interpretable, token-level attributions of psychological drivers in agent decisions.

Figure 6: Semantic landscape of the “Universalism” value; term sizes proportional to their relevance score as value activators.

Population-Level Value Dynamics

Analysis of population-level simulations (detailed in the supplementary figures) shows that previous approaches, when evaluated on all Schwartz value dimensions, suffer from mode collapse, losing the natural multimodality of human preference distributions. The CVA framework, in contrast, preserves the empirical variance and central tendency, tightly matching ground truth psychometric metrics and outperforming baselines across all group-level diversity and fidelity indices.

Theoretical and Practical Implications

The findings have several direct implications:

1. Evaluation: Establishes that existing "LLM-as-a-judge" or internal self-consistency evaluation regimes mask agent bias and overfit, necessitating empirically grounded external benchmarks such as CVABench.
2. Model Design: Demonstrates that modular architectures with explicit psychological reasoning constraints and value-driven discriminators yield not just higher-fidelity behavior, but also interpretable cognitive attributions. This enables robust behavioral alignment.
3. Psychometric Integration: Shows that the integration of generative psychometrics (e.g., GPV-based annotation) offers stable, scaling evaluation and supervision mechanisms, more robust than self-report or LLM-judgment alone.

Limitations and Future Directions

The CVA paradigm, while robust, is bounded by the scale and domain diversity of CVABench, and relies on the fidelity of value annotation tools for reliable supervision. Extension to more nuanced behaviors (e.g., cultural consumption, complex moral dilemmas) and stronger cross-domain generalization is required. Furthermore, although the architectural bias loop is mitigated by separating verifier and generator training and by leveraging ground truth signals, residual measurement or annotation bias may influence value-action alignment. Large-scale, real-world human subject experiments remain essential to fully validate agent generalizability.

Conclusion

The CVA architecture represents a substantive advance in value-driven LLM agents by operationalizing context-sensitive, dynamic value activation and decoupling action generation from value verification. Through comprehensive empirical evaluation and analytical benchmarks, the work demonstrates the failure of prompt-driven role-play paradigms to sustain human-like behavioral diversity, and establishes a rigorous framework for constructing interpretable, psychologically plausible agents. Its modular design and training protocol offer a blueprint for robust, transparent, and scalable human simulation in LLM-based architectures, with significant implications for both AI safety and HCI-driven agent design.