Clarifying Ambiguity in Language Models

• The paper introduces a task-agnostic framework that determines when to clarify ambiguous inputs by modeling latent intent entropy.
• It proposes a three-stage process: identifying when clarification is needed, generating a clarifying question, and refining responses using user feedback.
• Empirical evaluations on QA, machine translation, and NLI show performance gains by selectively clarifying high-aleatoric, low-epistemic uncertainty cases.

Searching arXiv for the target paper and closely related clarification work to ground the article. Resolving ambiguity through interaction is a core property of natural language use and a central requirement for conversational LLMs. “Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs” formalizes this problem as a task-agnostic framework in which a LLM must decide when clarification is needed, determine what clarifying question to ask, and then respond correctly after receiving the user’s answer (Zhang et al., 2023). The work studies this behavior across question answering, machine translation, and natural language inference, and introduces INTENT-SIM, an uncertainty estimation method that models ambiguity through entropy over user intents rather than treating all uncertainty alike (Zhang et al., 2023).

1. Problem formulation and motivation

The paper addresses a pervasive but narrowly defined problem: user inputs to LLMs are often ambiguous. A request such as “Who won the US Open?” or “Translate: It’s a little steeper than I was expecting” can admit multiple plausible interpretations, and human interlocutors normally resolve such ambiguity by asking clarifying questions rather than silently committing to one reading (Zhang et al., 2023). Current LLM-based assistants, by contrast, often either pick one interpretation without acknowledgment or hedge without actually resolving the ambiguity (Zhang et al., 2023).

The central objective is to make LLMs behave more like competent conversational partners: ask clarification when it matters, ask the right question, and use the answer to improve the final response (Zhang et al., 2023). This objective is motivated by a trade-off intrinsic to natural language. Ambiguity can be efficient because it allows speakers to rely on shared context, but when context is insufficient, additional interaction becomes necessary (Zhang et al., 2023). The paper therefore treats clarification not as a special-case repair strategy, but as a general mechanism for intent resolution in LM-based assistants.

A key conceptual distinction is drawn between epistemic and aleatoric uncertainty. Epistemic uncertainty arises when the model lacks the relevant knowledge; in such cases clarification may not help, because the model may still be unable to answer once the ambiguity is resolved. Aleatoric uncertainty arises when the input itself supports multiple plausible interpretations; in these cases clarification can materially improve correctness because the user can specify their intended meaning (Zhang et al., 2023). The “when to clarify” problem is explicitly defined as identifying examples with high aleatoric but low epistemic uncertainty (Zhang et al., 2023).

2. Three-stage clarification framework

The framework is organized as a task-agnostic, three-stage pipeline (Zhang et al., 2023). Each interaction begins with an ambiguous user request $x$, for which there may be multiple feasible outputs

$Y = \{y_i\}_{i=1}^k,$

with one true intended output $y^* \in Y$ (Zhang et al., 2023). The user intent is modeled as a latent distribution

$P(y = y^* \mid x),$

which is not directly observed but defines the ambiguity structure of the request (Zhang et al., 2023).

The first stage is when to clarify. The system learns an uncertainty score

$u(x) \in \mathbb{R}$

such that larger values indicate that clarification is likely to improve performance (Zhang et al., 2023). The second stage is what to ask. Given that clarification is warranted, the system generates a question $q$ that distinguishes among feasible outputs or their corresponding intent descriptions (Zhang et al., 2023). The third stage is how to respond after clarification. Once the user provides an answer $a^*$, the model uses $(x, q, a^*)$ to produce the final task-specific output (Zhang et al., 2023).

The paper assumes a bipartite matching between feasible outputs $Y$ and clarifying answers

$A = \{a_i\}_{i=1}^k,$

so that each answer $Y = \{y_i\}_{i=1}^k,$0 corresponds to exactly one feasible output $Y = \{y_i\}_{i=1}^k,$1, and vice versa (Zhang et al., 2023). Under this assumption, a successful clarifying question is one for which the user’s reply identifies the correct intent, allowing the model to recover $Y = \{y_i\}_{i=1}^k,$2.

Evaluation is task-specific at the final stage. The paper uses answer recall for question answering, label accuracy for natural language inference, and contrastive accuracy for machine translation (Zhang et al., 2023). This design is intended to separate the decision to clarify from the task-level utility gained once clarification occurs.

3. INTENT-SIM and entropy over user intents

The paper’s principal technical contribution is INTENT-SIM, an unsupervised uncertainty estimation method that predicts whether clarification will be useful by estimating entropy over user intents (Zhang et al., 2023). Its central intuition is straightforward. If a model expects that nearly all reasonable users would answer a clarifying question in the same way, then the input is effectively unambiguous; if it anticipates many different, comparably plausible answers, then the input is highly ambiguous and clarification is likely to help (Zhang et al., 2023).

INTENT-SIM operationalizes this in four steps: the model generates a clarifying question, samples many possible user answers to that question, clusters semantically equivalent answers so that paraphrases count as the same intent, and computes the entropy of the resulting cluster distribution (Zhang et al., 2023). That entropy becomes the uncertainty score $Y = \{y_i\}_{i=1}^k,$3.

The algorithm is presented with an LLM $Y = \{y_i\}_{i=1}^k,$4, an NLI model $Y = \{y_i\}_{i=1}^k,$5, user input $Y = \{y_i\}_{i=1}^k,$6, sampling temperature $Y = \{y_i\}_{i=1}^k,$7, and number of simulated answers $Y = \{y_i\}_{i=1}^k,$8, with $Y = \{y_i\}_{i=1}^k,$9 in the reported experiments (Zhang et al., 2023). The clarifying question is generated by greedy decoding: $y^* \in Y$0 The subsequent steps simulate possible user replies and induce an intent distribution through semantic clustering (Zhang et al., 2023).

This design is notable because it targets ambiguity from the model’s own perspective. Rather than using generic predictive uncertainty, it explicitly estimates variation over plausible user intents. That makes it especially suited to the paper’s stated goal of finding high-aleatoric, low-epistemic cases (Zhang et al., 2023). A plausible implication is that INTENT-SIM treats clarification as a selective intervention guided by latent intent structure rather than by confidence alone.

4. Experimental setting and empirical findings

The framework is evaluated on three NLP applications: question answering, machine translation, and natural language inference (Zhang et al., 2023). These tasks differ substantially in output space and supervision structure, which the paper uses to support the claim that clarification can be studied in a task-agnostic way (Zhang et al., 2023).

For the first subtask—determining when clarification is needed—the paper reports that INTENT-SIM consistently outperforms existing uncertainty estimation approaches at identifying predictions that benefit from clarification (Zhang et al., 2023). When the system is allowed to ask for clarification on only 10% of examples, it is able to double the performance gains over randomly selecting examples to clarify (Zhang et al., 2023). The method is also reported to be robust across a wide range of NLP tasks and LLMs (Zhang et al., 2023).

These findings support a more selective view of clarification. The paper does not advocate asking questions whenever ambiguity is possible; instead, it frames the problem as optimizing the utility of interruption. This aligns with later work that treats clarification as an ask-or-answer decision problem under underspecification, including visual question answering settings where clarification is explicitly modeled as a discrete control choice (Cao et al., 23 Jan 2026), and dialogue settings that connect clarification frequency to expected regret and action cost (Tsvilodub et al., 2 Feb 2026).

The paper’s empirical setup also implies a separation between the quality of the clarifying question and the utility of asking it at all. This is important because a system may be capable of generating fluent clarification questions while still failing to identify the inputs for which clarification is worth the interaction cost.

5. Relation to later clarification research

Subsequent work extends the clarification paradigm in several directions. In information retrieval, ambiguity-aware prompting methods classify query ambiguity into compact action-oriented types such as Semantic, Generalize, and Specify, using this intermediate structure to guide clarification generation (Tang et al., 16 Apr 2025). This suggests a complementary perspective: whereas INTENT-SIM estimates whether clarification will be useful by modeling entropy over intents, later IR work constrains the reasoning process for what kind of clarification should be asked (Tang et al., 16 Apr 2025).

In multimodal settings, later benchmarks show that ambiguity can depend on visual context rather than purely textual underspecification. VAGUE evaluates multimodal intention disambiguation where the intended action is only apparent from the accompanying image, and reports that existing multimodal models remain far below human performance (Nam et al., 2024). Related work on context-dependent VQA treats the correct answer as a function $y^* \in Y$1 of image, question, and missing external context, and uses a Clarify-or-Answer controller to decide whether a clarifying question is needed (Cao et al., 23 Jan 2026). These lines of work expand the original paper’s concern with latent intent into settings where ambiguity is distributed across modalities or external context.

A different extension separates clarification from abstention. Belief-Augmented Generation formulates three strategies—direct answer, clarification question, and abstain—by prompting a model to reason over a sampled belief state, and reports that LLMs by default rarely clarify or abstain even when uncertainty is visible in their sampled outputs (Baan et al., 25 May 2026). This suggests that the original “when to clarify” problem may be part of a broader strategic routing problem in which ambiguity-driven clarification and knowledge-driven abstention must be disentangled.

In software and code generation, later systems operationalize clarification before action in highly constrained domains. ProCAD uses a proactive clarifying agent to audit text-to-CAD prompts for ambiguous dimensions, conflicting dimensions, and geometrically impossible dimensions before generating CadQuery programs (Yuan et al., 3 Feb 2026). ClarifyCoder similarly trains code LLMs to ask for clarification rather than immediately emit code when requirements are ambiguous, incomplete, or inconsistent (Wu et al., 23 Apr 2025). These systems instantiate the paper’s general thesis in settings where underspecification has direct execution-time consequences.

6. Conceptual significance and limitations

The paper’s main significance lies in treating clarification as an explicit computational object rather than an informal conversational nicety. It decomposes the problem into when to clarify, what to ask, and how to use the resulting answer, and it locates the first of these in entropy over latent user intents (Zhang et al., 2023). This is conceptually distinct from generic uncertainty estimation, because it isolates the subset of uncertainty that interaction can resolve.

At the same time, the framework depends on simplifying assumptions. The bipartite matching between feasible outputs and clarifying answers gives a clean formalization, but real conversations may involve non-bijective mappings, partial clarifications, or user replies that themselves introduce new ambiguity (Zhang et al., 2023). Later multi-turn benchmarks explicitly study such interactional complications, including vague, contradictory, off-focus, and factually wrong user replies, and report that performance degrades as dialogue depth increases (Luo et al., 24 Dec 2025). This suggests that the original framework is strongest as a foundational single-clarification formalization rather than as a full theory of conversational repair.

Another limitation is that the paper’s central contribution is concentrated on the first subtask. The abstract and details emphasize INTENT-SIM and the utility of selective clarification, but provide less comparable emphasis on a unified optimization of question generation and post-clarification response (Zhang et al., 2023). A plausible implication is that the framework establishes a decomposition that later work can specialize by domain: ambiguity-aware prompting in IR (Tang et al., 16 Apr 2025), reward-based question design in social reasoning (Pyatkin et al., 2022), or structured agentic control in multi-turn dialogue (Luo et al., 24 Dec 2025).

Despite these limits, the paper lays a foundation for studying clarifying interactions with LLMs by defining a shared formal vocabulary and by empirically demonstrating that clarification utility can be predicted more effectively when ambiguity is modeled as entropy over user intents rather than as undifferentiated uncertainty (Zhang et al., 2023).