Uncertainty Decomposition for Clarification Seeking in LLM Agents

Abstract: Recent position papers argue that the classical aleatoric/epistemic uncertainty framework is insufficient for interactive LLM agents and call for underspecification-aware, decomposed, and communicable uncertainty representations that can unlock new agent capabilities such as proactive clarification seeking and shared mental-model building. Practical deployment constraints -- black-box APIs, interactive latency budgets, and the absence of labeled trajectories -- rule out logprob-based, multi-sampling, and training-based methods, leaving prompt-based estimation as the most viable family for surfacing such signals at deployment time. We answer this call with a simple prompt-based decomposition that separates action confidence from request uncertainty (u), enabling the agent to ask for clarification when the task specification is ambiguous. To evaluate it, we introduce two clarification-augmented benchmarks (WebShop-Clarification and ALFWorld-Clarification) in which 50% of tasks are deliberately underspecified, and systematically compare the proposed decomposition against ReAct+UE and Uncertainty-Aware Memory (UAM) across five LLM backbones (GPT-5.1, DeepSeek-v3.2-exp, GLM-4.7, Qwen3.5-35B, GPT-OSS-120B) on these variants together with the standard WebShop, ALFWorld, and REAL benchmarks for fault detection. Averaged across the five backbones, the proposed decomposition improves clarification F1 on ALFWorld-Clarification by 73% over ReAct+UE and by 36% over UAM, and leads clarification F1 on every backbone on WebShop-Clarification and on four of five backbones on ALFWorld-Clarification, indicating that the gains generalize beyond a single LLM.

• The paper introduces a prompt-only method that separates action confidence from request uncertainty, enabling LLM agents to ask for clarification when goals are underspecified.
• The decomposition improves clarification F1 by 73% over ReAct+UE and 36% over UAM on ALFWorld-Clarification, while leading on every WebShop-Clarification backbone.
• The results show that clarification gains come with capability dilution, overconfident scores, and aggregation sensitivity, highlighting the need for calibration and trained uncertainty mechanisms.

Motivation and problem statement

Interactive LLM agents operate under partial observability: they receive underspecified natural-language goals, observe noisy environment states, and chain multiple reasoning steps, so early misinterpretations can propagate into confidently wrong outcomes. Recent position papers argue that the classical aleatoric/epistemic dichotomy is inadequate in this setting and call for underspecification-aware, decomposed, and communicable uncertainty representations that enable proactive behaviors such as clarification seeking. This paper answers that call with a practical constraint: black-box API access, interactive latency budgets, and the absence of labeled trajectories jointly rule out logprob-based, multi-sampling, training-based, and white-box methods. The author argues—via a systematic taxonomy of method families—that prompt-based estimation is the only family requiring neither logprob access nor model internals nor multiple inference passes per step, and restricts the experimental comparison accordingly.

The core observation motivating the method is that existing prompt-based approaches emit a single scalar confidence per step, conflating two distinct sources of uncertainty: difficulty of the action given a well-specified goal (which warrants cautious proceeding) versus ambiguity of the goal itself (which warrants asking the user). A single score cannot distinguish these cases.

Proposed method

The proposed decomposition extends Uncertainty-Aware Memory (UAM) by replacing its single confidence scalar with two signals emitted in one forward pass:

• Action confidence ($c_t$): likelihood that the chosen action progresses toward task completion, conditioned on the current understanding of the goal.
• Request uncertainty ($u_t$): degree to which the user's goal is underspecified, on an anchored three-point scale (0 = fully specified; 0.5 = at least one parameter left open; 1 = critical details missing).

A deterministic routing rule triggers the special request action when $u_t \ge \theta$ (default $\theta = 0.5$), which makes clarification seeking evaluable as binary classification against a ground-truth underspecification label $z$. Both signals, with free-text explanations, are propagated through the interaction history so later steps can reason over accumulated request-level uncertainty. Notably, $u_t$ is assessed before the action is emitted, ensuring underspecification is caught prior to acting.

Benchmarks and evaluation protocol

Two clarification-augmented variants are introduced: WebShop-Clarification (attribute words and option clauses stripped from 50% of instructions) and ALFWorld-Clarification (object or receptacle randomly removed from 50% of goals). When the agent requests clarification on an underspecified task, the fully specified goal is revealed and the episode continues, allowing fair measurement of task success. Evaluation covers three methods (ReAct+UE, UAM, and the proposed decomposition) across five backbones (GPT-5.1, DeepSeek-v3.2-exp, GLM-4.7, Qwen3.5-35B, GPT-OSS-120B) on five benchmark configurations, with 100 tasks per (method, benchmark) pair. Two metric families are reported: fault detection (ROC-AUC, ECE, Brier over trajectory-level aggregated scores) and clarification seeking (precision/recall/F1/accuracy of the clarification decision).

Results

Clarification seeking. Averaged across the five backbones, the decomposition improves clarification F1 on ALFWorld-Clarification by 73% over ReAct+UE and 36% over UAM, leading F1 on every backbone on WebShop-Clarification and four of five backbones on ALFWorld-Clarification. Representative gains are large: GLM-4.7 reaches F1 of 0.756 versus 0.405 (ReAct+UE) on WebShop-Clarification, and GPT-5.1 reaches precision of 0.938 on ALFWorld-Clarification. These results indicate the benefit generalizes beyond a single backbone rather than reflecting model-specific behavior.

Fault detection. The decomposition does not sacrifice the conventional objective: it attains the highest last- and average-aggregation ROC-AUC on WebShop and REAL for GPT-5.1 and remains within 0.08 of the best on ALFWorld, performing on par with both baselines across backbones.

Capability dilution. Mean success rate falls monotonically from 28.6% (ReAct+UE) to 27.8% (UAM) to 27.0% (proposed). Because UAM and the proposed method differ only in the added request-uncertainty signal, the 0.8-point gap isolates the cost of the enlarged instrumentation. The degradation concentrates on demanding benchmarks (REAL: 32.1% → 19.2%; WebShop: 31.9% → 26.1%) and is absent on structured ALFWorld (57.5% → 58.9%). The author attributes this to a fixed shared reasoning budget split across task-solving, confidence estimation, and request assessment—a limitation not resolvable by prompt engineering within the prompt-only regime.

Overconfidence. Reliability diagrams show all curves below the diagonal for every method, backbone, and benchmark, with per-panel ECE ranging from 0.24 to 0.66. Verbalized confidence is therefore useful as a ranking signal but cannot be read as a probability without post-hoc recalibration.

Aggregation as hidden hyperparameter. A diagnostic replaces real per-step confidences with i.i.d. $\mathcal{U}(0,1)$ draws or a deterministic $1/\text{length}$ score under product aggregation. On ALFWorld both confidence-free surrogates reach ROC-AUC of 0.92–0.99, matching or exceeding real-product scores for nearly every method. Since failed trajectories are systematically longer there and geometric means shrink with trajectory length, high product-aggregation ROC-AUC reflects length confounding rather than informative confidence—the paper explicitly cautions against interpreting such scores as evidence of signal quality.

Threshold sensitivity. No single $\theta$ dominates all (model, benchmark) pairs; $\theta = 0.25$ yields the highest mean F1 averaged over backbones but differs from $u_t$0 by at most 0.03, which is reported as a balanced default.

Limitations

Three limitations are conceded directly. First, capability dilution: each additional uncertainty objective degrades primary-task performance, rooted in a shared reasoning budget. Second, systematic overconfidence affects all prompt-based methods structurally, since an agent committed to an action has an incentive to justify rather than critique it. Third, trajectory-level aggregation choice can dominate the effect of the uncertainty method itself, and product aggregation can act as a length proxy. Additionally, the clarification-augmented benchmarks rely on procedurally generated underspecification and score clarification as a single binary action, leaving open whether the gains transfer to organically ambiguous tasks or to multi-turn clarification quality.

Conclusion

This paper contributes a prompt-only decomposition of agentic uncertainty into action confidence and request uncertainty, two clarification-augmented benchmarks, and a systematic cross-backbone comparison within the prompt-based family. The decomposition substantially improves clarification F1 while preserving fault-detection performance, demonstrating that separating goal ambiguity from task difficulty enables proactive clarification where scalar-confidence baselines fail. At the same time, capability dilution, overconfidence, and aggregation sensitivity are presented as fundamental to the prompt-based paradigm, and the author argues the decomposition should ultimately be moved into the model via alignment-based training (RLHF, DPO, GRPO), following evidence that fine-tuned clarifiers achieve far larger gains.