---
title: Dual-Process Agentic UQ Framework
url: https://www.emergentmind.com/topics/dual-process-agentic-uq-auq
type: topic
---

# Dual-Process Agentic UQ Framework

Dual-Process Agentic Uncertainty Quantification (AUQ) is a rigorous framework for transforming uncertainty assessment in intelligent agents from passive monitoring to active, bi-directional control. By decomposing agentic reasoning into coupled inference and control processes, AUQ enables more reliable long-horizon decision-making, calibrates both intrinsic and inherited uncertainty, and situates uncertainty as a first-class driver of agentic policy and exploration. Instantiations span large language model (LLM) agents in sequential and multimodal reasoning, social-psychological dual-process models, and conformalized tool calibration in vision–language systems, providing a unifying perspective on robust agent design.

## 1. Motivations and Foundations

Traditional uncertainty quantification (UQ) in AI agents, especially those powered by LLMs, primarily focuses on single-turn predictions, using metrics such as predictive entropy or token-level variance to detect low-confidence outputs. However, in sequential, agentic contexts—where decisions propagate over multiple steps—these approaches are fundamentally insufficient. AUQ frameworks address the compounding and propagation of epistemic risk, often termed the “Spiral of Hallucination,” wherein small early errors propagate irreversibly through the trajectory, substantially degrading reliability [2601.15703], [2506.17419].

The dual-process perspective, rooted in both cognitive psychology and computational reinforcement learning, dichotomizes agent cognition into two tightly coupled subsystems: a fast, heuristic stream (often memory- or affect-driven) and a slow, analytic control mechanism. This separation underpins a spectrum of modern agentic UQ algorithms for multimodal, language, and social reasoning tasks [2601.15703], [2503.08308], [1908.03106].

## 2. Dual-Process AUQ Architectures

### 2.1 System 1 and System 2 in AUQ

The System 1 (“fast path”) provides uncertainty-aware implicit control, typically by propagating verbalized confidence or heuristic cues through memory and attention. In agentic LLM contexts, this is instantiated as Uncertainty-Aware Memory (UAM): each output is augmented with a scalar confidence $\hat{c}_t$ and a natural-language explanation $\hat{e}_t$; both are persistently retained in the agent’s memory and context window, biasing subsequent inference away from overcommitment [2601.15703].

System 2 (“slow path”) is invoked when confidence falls below a threshold, triggering Uncertainty-Aware Reflection (UAR): targeted, high-cost deliberation (e.g., best-of-N sampling or reflective planning) guided by prior uncertainty explanations $\hat{e}_t$. System 2 selects the final decision via a consistency-weighted aggregation across reflective candidates, only incurring computational cost when confidence deficits arise [2601.15703].

### 2.2 Dual-Process in Information-Theoretic AUQ

In sequence modeling, notably with LLMs, dual-process AUQ divides total predictive uncertainty at each step into an intrinsic component (local entropy given past actions) and an extrinsic component (mutual information with previous decisions). This decomposition allows agents to explicitly track how much risk is “inherited” along the trajectory and pre-allocate attention/resources [2506.17419].

### 2.3 Dual-Process in Multimodal and Social Agents

In vision–language models, agentic decision loops (dynamic region-of-interest selection) are paired with conformal prediction (CP)-calibrated tool outputs. The agent only attends to regions deemed relevant by stepwise reasoning (fast path), while CP-based calibration (slow path) ensures strict coverage guarantees regardless of tool miscalibration [2503.08308]. In social-psychological agents (BayesAct), the streams correspond to affective (connotative) and decision-theoretic (denotative) processes, with somatic coherence ensuring coherence and adaptive policy weighting [1908.03106].

## 3. Mathematical Formalism and Uncertainty Metrics

### 3.1 Propagated Uncertainty in Sequential Decision-Making

Given a trajectory $\mathcal{P} = (Y_1, ..., Y_T) \sim p(\mathcal{P} \mid x)$:
$$
H(Y_t \mid x) = H(Y_t \mid Y_{1:t-1}, x) + \sum_{i=1}^{t-1} I(Y_t; Y_i \mid Y_{i+1:t-1}, x)
$$
- Intrinsic Uncertainty (IU): $H(Y_t \mid Y_{1:t-1}, x)$
- Extrinsic Uncertainty (EU): $\sum_{i=1}^{t-1} I(Y_t; Y_i \mid \ldots)$

The UProp estimator trades direct, intractable marginalization for trajectory-wise Pointwise Mutual Information (PMI) approximations via Monte Carlo sampling. For a sampled trajectory, IU is estimated with predictive entropy, and EU is approximated using kernel-smoothed PMI scores over per-step samples [2506.17419].

### 3.2 Calibration and Selection Metrics

In the System 1/2 setting [2601.15703]:
- The agent maintains a memory $M_t = \{ (o_i, a_i, \hat{c}_i, \hat{e}_i) \}_{i=0...t-1}$.
- For confidence aggregators $C(\tau)$, overall quality is $C_{\text{avg}} = \frac{1}{T} \sum \hat{c}_t$, and process reliability is $C_{\min} = \min_t \hat{c}_t$.
- Calibration metrics include Trajectory-ECE, Trajectory Brier Score, and AUROC for correct/incorrect trajectory discrimination.

In conformal prediction calibration [2503.08308]:
- The coverage guarantee enforces $P(\text{true label} \in R) \ge 1 - \alpha$ for calibrated tool outputs.
- MLLM output uncertainty is quantified as $US(A) = (1/T) \sum_{t=1}^T k_t$, where $k_t$ is the minimal token set covering top-$p$ mass at decoding step $t$.

### 3.3 Dual-Process Policy Switching

AUQ policies select between forward/fast (System 1) and reflective/slow (System 2) passes adaptively:
$$
\pi_{\text{dual}}(a | h_t) = 
\begin{cases}
\pi_{\text{fwd}}(a|h_t, M_t) & \text{if } \hat{c}_t \ge \tau \\
\pi_{\text{inv}}(a|h_t) & \text{if } \hat{c}_t < \tau
\end{cases}
$$

When denotative entropy is low, action selection is purely instrumental; when high, affective deflection-minimizing or heuristics dominate [1908.03106].

## 4. Algorithmic Implementations and Pseudocode

A core feature of practical AUQ frameworks is training-free deployment. All logic is embedded via prompt engineering, context manipulation, and selection wrappers.

### Example AUQ Step (per [2601.15703])

```python
# Pseudo-code for one AUQ step
Prompt model with M_t and o_t -> receive (â_t, ĥc_t, ĥe_t)
if ĥc_t >= τ:
    execute a_t = â_t
else:
    construct reflection prompt with ĥe_t
    sample N candidates
    select via consistency-weighted score
append (o_t, a_t, ĥc_t, ĥe_t) to M_t
step environment
```

### UProp TDP Sampling (per [2506.17419])

```python
for z in range(Z):  # Number of trajectories
    for t in range(1, T_z+1):
        # MC sample N continuations at step t
        y_t_samples = [sample_from_p(y_t | y_{1:t-1}, x) for _ in range(N)]
        # Compute IU_t as average negative log-probability
        IU_t = -np.mean([log_p(y) for y in y_t_samples])
        # Estimate PMI over kernel between y_{t-1} for all previous steps
# Aggregate IU and PMI, normalize, average over Z
```

## 5. Applications and Empirical Findings

AUQ has been empirically validated across diverse domains:

- **Closed-loop planning and open-ended research**: In ALFWorld, WebShop, and DeepResearch Bench, Dual-Process AUQ achieves substantial improvements in both success rate and trajectory calibration compared to single-turn or naive ensembles, e.g., ALFWorld success rate increases from 63.6% (ReAct) to 74.3% (Dual-Process), and end-state AUROC improves from 0.913 to 0.968 [2601.15703].
- **Multimodal reasoning**: The SRICE agent achieves an average 4.6% improvement over base MLLM performance across five datasets, outperforming some finetuning-based approaches [2503.08308].
- **Uncertainty aggregation**: UProp’s explicit separation of intrinsic and extrinsic uncertainty in agentic LLMs yields AUROC gains (e.g., 0.771 on AgentBench-OS vs. 0.748 for the best baseline) and boosts selective prediction reliability in safety-critical multi-step agents [2506.17419].
- **Social affective decision-making**: The BayesAct dual-process model unifies both affective-alignment and utility-maximization, providing a mathematically grounded approach to RL exploration/exploitation unification and social conformity [1908.03106].

## 6. Limitations, Open Problems, and Prospective Directions

Known limitations of current AUQ frameworks include reliance on LLMs’ ability to verbalize well-calibrated confidences (which may degrade in smaller models), computational overheads of adaptive reflection and MC sampling, and heuristic aspects in mutual information estimation and contextual similarity measures [2601.15703], [2506.17419]. In multimodal systems, conformal calibration guarantees are limited to the finite-sample regime and assume sensible calibration datasets [2503.08308].

Several open directions have emerged:
- Adaptive risk budgeting for per-step $\tau$ selection and meta-controllers to tune reflection frequency.
- Learned or meta-learned similarity kernels for PMI approximations in UProp.
- Extensions to continuous-action domains and tighter theoretical error bounds on information-theoretic UQ.
- Integration of affective alignment principles with explicit statistical UQ in multi-modal and agentic RL systems [1908.03106].

## 7. Synthesis and Theoretical Significance

Dual-Process Agentic Uncertainty Quantification establishes a general, rigorous foundation for decision-aware UQ in agents operating over long-horizon, context-propagating tasks. By tightly integrating fast, memory-based uncertainty propagation with slow, targeted reflection triggered by explicit confidence deficits, it addresses the compounding risk inherent in agentic sequences. The framework unites formal tools from information theory, calibration statistics, and Bayesian affective modeling, and demonstrates broad empirical gains in both performance and reliability across agentic, multimodal, and social-psychological AI systems [2601.15703], [2506.17419], [2503.08308], [1908.03106].

| Instantiation              | Fast Path: System 1                  | Slow Path: System 2                      |
|----------------------------|--------------------------------------|------------------------------------------|
| Generic LLM AUQ [2601.15703]| UAM: verbalized confidence/explanation| UAR: reflection invoked on low-confidence|
| UProp [2506.17419]         | Intrinsic uncertainty (IU)           | Extrinsic MI-based uncertainty (EU)      |
| SRICE [2503.08308]         | Agentic RoI selection, CoT loop      | CP-based tool calibration                |
| BayesAct [1908.03106]      | Affective alignment (connotative)    | Decision-theoretic/utility maximization  |

Dual-process AUQ thus frames uncertainty as both a continuous control statistic and a selective reflection trigger, providing a scalable and theoretically grounded solution to reliability in modern agentic AI.

Source: https://www.emergentmind.com/topics/dual-process-agentic-uq-auq