---
title: Value-Guided Construal Models
url: https://www.emergentmind.com/topics/value-guided-construal-vgc-models
type: topic
---

# Value-Guided Construal Models

A Value-Guided Construal (VGC) model is a theoretical and algorithmic framework for adaptive representation, decision, or generation, in which the construction or selection of internal representations, policies, or outputs is explicitly optimized under task-specific value functions and resource constraints. Originating from resource-rational and bounded rationality perspectives, VGC formalizes how agents—biological or artificial—simplify complex environments, balance utility against representational cost, or dynamically steer large generative models, by learning value functions or explicit construal policies that guide which information enters into planning, inference, or output construction. In recent years, a diverse literature spanning computational cognitive science, model-based control, and large language model alignment has instantiated the VGC approach in domains ranging from human-like mental simulation to multi-objective LLM decoding and goal-conditioned world-model planning.

## 1. Formal Foundations of Value-Guided Construal

VGC models share a canonical structure: they define a trade-off between the utility of a representation, policy, or output—usually quantified by a value or reward function—and its cost, complexity, or resource consumption. The general objective is  
$$
r^* = \arg\max_{r \in \mathcal{R}} \left[U(r) - \beta C(r)\right]
$$  
where $r$ is a representation (or construal) selected from space $\mathcal{R}$, $U(r)$ is expected task utility under $r$, $C(r)$ is a cost or complexity measure (e.g., $|r|$, KL divergence, coding length), and $\beta$ is a trade-off parameter. This abstraction includes:  
- **Perceptual/Planning Representational VGC**: $r$ is a subset of environment features, obstacles, or object encodings as in maze or grid-world planning [2506.09520, 2601.14514].
- **Value-Weighted Language Model Decoding**: $r$ as a sequence prefix or candidate output, with utility defined via external or learned reward models, and cost as divergence from a pretrained model [2508.13415, 2503.02368].
- **Goal-Conditioned World Modeling**: $r$ as a state embedding for planning, with value defined by negative cost-to-go or embedding distance, cost as representational mismatch [2601.00844].

Notably, VGC is not tied to any particular format of value function or resource cost, allowing both soft-inclusion (e.g., smoothed by attentional kernels [2506.09520]) and discrete selection.

## 2. Value-Guided Decoding and Inference in Language Models

Recent applications of VGC in LLMs optimize over output sequences using learned value functions to steer generation towards user-specified objectives without full retraining.

### Multi-Objective Alignment (MAVIS)

MAVIS [2508.13415] trains a set of per-objective value models $\{V_m\}$—each a lightweight LM with a regression head—to estimate KL-regularized expected returns. At inference, user weights $\lambda_m$ induce a tilting function:
$$
p_{\text{tilt}}(y|x;\lambda) \propto p_\theta(y|x)\;\exp\!\Bigl(\frac{1}{\eta}\sum_{m}\lambda_m r_m(y|x)\Bigr)
$$
Token-level policies are adjusted as:
$$
\pi_{\text{MAVIS}}(a_t|s_t) \propto \pi_{\mathrm{ref}}(a_t|s_t) \exp\Bigl(\beta \sum_m \lambda_m V_m(s_t \oplus a_t)\Bigr)
$$
Each $V_m$ is trained by KL-regularized policy iteration, where empirical returns penalized by log-probability ratios are regressed onto value heads. This enables post hoc adjustment of output tradeoffs among multiple goals, strict monotonic policy improvement, and expansion of the achievable Pareto frontier relative to baseline mixtures.

### Iterative Value Function Optimization (IVO)

IVO [2503.02368] introduces a critic $V_\theta$ trained with Monte Carlo rollouts and regression, and iteratively improves the policy via:
$$
\pi_V(a|s) \propto \pi_{\text{base}}(a|s)\, e^{\beta V_\theta(f(s,a))}
$$
This approach allows steering of decoding to maximize reward without updating LLM backbone weights, substantially reducing computational cost relative to RLHF. IVO achieves significant empirical gains on summarization, dialog, and instruction tasks, dominates prior value-guided sampling methods (FUDGE, ARGS, VAS), and yields favorable GPT-4 win-rates.

## 3. Value-Guided Construal in Human Planning and Mental Simulation

VGC has been applied to models of human planning, exemplifying the resource-rational principle that agents filter and encode only task-relevant features.

### Just-in-Time (JIT) World Modeling

JIT planning [2601.14514] implements VGC not via explicit search over representations, but through an interleaved simulate–lookahead–encode process. The agent maintains a working memory (construal) $C_t$ containing only a small subset $C^*\subseteq O$ of all possible objects or obstacles.
- Simulation steps trigger a lookahead that identifies unencoded but soon-to-be-relevant objects.
- Objects flagged are dynamically encoded; unused items decay probabilistically according to power-law forgetting.
- The process supports efficient prediction and planning with high correlation to human behavioral probes, reducing average objects represented and matching or exceeding classical VGC models in variant tasks.

Efficiency is obtained by estimating need probabilities for each object via Monte Carlo over sampled trajectories, updating construals "just in time" as demanded by the evolving simulation state.

### Attentional and Perceptual Modulation

Extensions incorporating visuospatial attention ("spotlight-VGC") [2506.09520] introduce soft gating over which features enter the task representation, parameterized by spatial kernels or lateralization, tuned via participant-specific attention radius. The agent's attention function $\alpha(s)$ influences which environmental features are included in the simplified model via smoothed inclusion probabilities, accounting for human-like crowding and lateralization effects in virtual maze navigation.

## 4. VGC in Moral Reasoning with Language Models

VGC is also employed for moral and value-sensitive LLMs [2506.14948]. Here, the construal is instantiated as a combination of structured prompts reflecting value systems and ethical theories, eliciting chain-of-thought-style justifications and decisions.
- A taxonomy of prompts combines psychological value frameworks (e.g., Schwartz, Moral Foundations) and explicit ethical theories (e.g., Care Ethics) to scaffold model reasoning.
- A distillation pipeline transfers competence from large teacher models, minimizing a hybrid loss over token-by-token imitation and semantic consistency, yielding scalable, interpretable, and value-grounded reasoning in small models.
- Structured prompting and distillation yield consistent improvements in moral decision accuracy and justification coherence over label-only baselines.

## 5. VGC in Goal-Conditioned World Models and Control

In model-based control, VGC formalizes how value structure shapes representation and action planning.

### JEPA World Models

Destrade et al. [2601.00844] introduce VGC within a Joint-Embedded Predictive Architecture (JEPA) for goal-reaching tasks.
- The value function $V^*(s;g)$—negative cost-to-go for reaching goal $g$—is approximated by $-d(\mathcal{E}_\theta(s),\,\mathcal{E}_\theta(g))$, with $d$ a Euclidean or quasi-metric distance in embedding space.
- Training alternates or jointly optimizes a JEPA prediction loss and an Implicit Q-Learning (IQL) value loss, with expectile regression ($L_\tau^2$) shaping the embedding geometry for effective planning.
- Model Predictive Path Integral control (MPPI) uses these distances at test time for high-accuracy action planning, outperforming contrastive and standard regression approaches, but displays limitations in long-range calibration and stochastic settings.

## 6. Algorithmic Summaries and Theoretical Guarantees

Across VGC instantiations, several recurrent themes and guarantees emerge:

| Domain              | Value Function                  | Inference/Planning Mechanism                   | Theoretical Guarantee         |
|---------------------|--------------------------------|-----------------------------------------------|------------------------------|
| LLM Decoding        | KL-regularized, per-objective  | Exponential tilting (MAVIS), IVO top-k/beam   | Monotonic improvement, Pareto optimality [2508.13415, 2503.02368] |
| Human Planning      | Utility vs. complexity          | Greedy/spotlight representational search, JIT  | Efficiency, tight human fits [2601.14514, 2506.09520]   |
| World Modeling      | Negative embedding distance     | MPPI w/ JEPA, value-influenced control        | Improved planning accuracy [2601.00844]  |
| Moral Reasoning     | Prompt-structured value tradeoff| Structured prompting, distillation            | Accuracy, coherence improvements [2506.14948] |

KL-regularized policy iteration in LLM contexts enjoys strict monotonic policy improvement, provable convergence to optimal token-level policies under certain bandit settings, and empirical Pareto-front dominance in multi-objective evaluation [2508.13415]. In perceptual VGC, resource-bounded optimization yields fit measures (e.g., human–model correlation, RMSE, log-likelihood) that closely match human data [2601.14514, 2506.09520].

## 7. Limitations, Efficiency Gains, and Open Challenges

VGC models achieve substantial empirical and computational efficiency over traditional RLHF or brute-force search:
- MAVIS and IVO require only small value heads and limited rollout sampling, yielding speedups of $\gtrsim$100$\times$ for LLM alignment [2508.13415, 2503.02368].
- JIT and attentional VGC in perceptual domains encode significantly fewer features for similar predictive power, trading occasional planning suboptimality for memory savings [2601.14514].
- In JEPA world models, value-guided construals improve planning accuracy but struggle with rare state–goal pairs and with calibration far from the goal; improvements require either hierarchical latents or more strategically curated datasets [2601.00844].

Broader open issues include:
- Extending VGC to domains with high non-stationarity or combinatorial construal spaces.
- Scalability of representational search in environments with ambiguous or weakly-structured value signals.
- Formal generalization bounds under resource constraints and finite-sample regimes.

---

References:  
- MAVIS: Multi-Objective Alignment via Value-Guided Inference-Time Search [2508.13415]  
- Iterative Value Function Optimization for Guided Decoding [2503.02368]  
- "Just in Time" World Modeling Supports Human Planning and Reasoning [2601.14514]  
- How attention simplifies mental representations for planning [2506.09520]  
- Structured Moral Reasoning in Language Models [2506.14948]  
- Value-guided action planning with JEPA world models [2601.00844]

Source: https://www.emergentmind.com/topics/value-guided-construal-vgc-models