---
title: Clarifying Ambiguity in Language Models
url: https://www.emergentmind.com/topics/clarify
type: topic
---

# Clarifying Ambiguity in Language Models

Searching arXiv for the target paper and closely related clarification work to ground the article.
Resolving ambiguity through interaction is a core property of natural language use and a central requirement for conversational language models. “Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs” formalizes this problem as a task-agnostic framework in which a language model must decide when clarification is needed, determine what clarifying question to ask, and then respond correctly after receiving the user’s answer [2311.09469]. The work studies this behavior across question answering, machine translation, and natural language inference, and introduces INTENT-SIM, an uncertainty estimation method that models ambiguity through entropy over user intents rather than treating all uncertainty alike [2311.09469].

## 1. Problem formulation and motivation

The paper addresses a pervasive but narrowly defined problem: user inputs to language models are often ambiguous. A request such as “Who won the US Open?” or “Translate: It’s a little steeper than I was expecting” can admit multiple plausible interpretations, and human interlocutors normally resolve such ambiguity by asking clarifying questions rather than silently committing to one reading [2311.09469]. Current LLM-based assistants, by contrast, often either pick one interpretation without acknowledgment or hedge without actually resolving the ambiguity [2311.09469].

The central objective is to make language models behave more like competent conversational partners: ask clarification when it matters, ask the right question, and use the answer to improve the final response [2311.09469]. This objective is motivated by a trade-off intrinsic to natural language. Ambiguity can be efficient because it allows speakers to rely on shared context, but when context is insufficient, additional interaction becomes necessary [2311.09469]. The paper therefore treats clarification not as a special-case repair strategy, but as a general mechanism for intent resolution in LM-based assistants.

A key conceptual distinction is drawn between epistemic and aleatoric uncertainty. Epistemic uncertainty arises when the model lacks the relevant knowledge; in such cases clarification may not help, because the model may still be unable to answer once the ambiguity is resolved. Aleatoric uncertainty arises when the input itself supports multiple plausible interpretations; in these cases clarification can materially improve correctness because the user can specify their intended meaning [2311.09469]. The “when to clarify” problem is explicitly defined as identifying examples with high aleatoric but low epistemic uncertainty [2311.09469].

## 2. Three-stage clarification framework

The framework is organized as a task-agnostic, three-stage pipeline [2311.09469]. Each interaction begins with an ambiguous user request \(x\), for which there may be multiple feasible outputs
\[
Y = \{y_i\}_{i=1}^k,
\]
with one true intended output \(y^* \in Y\) [2311.09469]. The user intent is modeled as a latent distribution
\[
P(y = y^* \mid x),
\]
which is not directly observed but defines the ambiguity structure of the request [2311.09469].

The first stage is **when to clarify**. The system learns an uncertainty score
\[
u(x) \in \mathbb{R}
\]
such that larger values indicate that clarification is likely to improve performance [2311.09469]. The second stage is **what to ask**. Given that clarification is warranted, the system generates a question \(q\) that distinguishes among feasible outputs or their corresponding intent descriptions [2311.09469]. The third stage is **how to respond after clarification**. Once the user provides an answer \(a^*\), the model uses \((x, q, a^*)\) to produce the final task-specific output [2311.09469].

The paper assumes a bipartite matching between feasible outputs \(Y\) and clarifying answers
\[
A = \{a_i\}_{i=1}^k,
\]
so that each answer \(a_i\) corresponds to exactly one feasible output \(y_i\), and vice versa [2311.09469]. Under this assumption, a successful clarifying question is one for which the user’s reply identifies the correct intent, allowing the model to recover \(y^*\).

Evaluation is task-specific at the final stage. The paper uses answer recall for question answering, label accuracy for natural language inference, and contrastive accuracy for machine translation [2311.09469]. This design is intended to separate the decision to clarify from the task-level utility gained once clarification occurs.

## 3. INTENT-SIM and entropy over user intents

The paper’s principal technical contribution is INTENT-SIM, an unsupervised uncertainty estimation method that predicts whether clarification will be useful by estimating entropy over user intents [2311.09469]. Its central intuition is straightforward. If a model expects that nearly all reasonable users would answer a clarifying question in the same way, then the input is effectively unambiguous; if it anticipates many different, comparably plausible answers, then the input is highly ambiguous and clarification is likely to help [2311.09469].

INTENT-SIM operationalizes this in four steps: the model generates a clarifying question, samples many possible user answers to that question, clusters semantically equivalent answers so that paraphrases count as the same intent, and computes the entropy of the resulting cluster distribution [2311.09469]. That entropy becomes the uncertainty score \(u(x)\).

The algorithm is presented with an LLM \(M\), an NLI model \(N\), user input \(x\), sampling temperature \(T\), and number of simulated answers \(S\), with \(S = 10\) in the reported experiments [2311.09469]. The clarifying question is generated by greedy decoding:
\[
q \leftarrow \text{GreedySample}(M, [x]).
\]
The subsequent steps simulate possible user replies and induce an intent distribution through semantic clustering [2311.09469].

This design is notable because it targets ambiguity from the model’s own perspective. Rather than using generic predictive uncertainty, it explicitly estimates variation over plausible user intents. That makes it especially suited to the paper’s stated goal of finding high-aleatoric, low-epistemic cases [2311.09469]. A plausible implication is that INTENT-SIM treats clarification as a selective intervention guided by latent intent structure rather than by confidence alone.

## 4. Experimental setting and empirical findings

The framework is evaluated on three NLP applications: question answering, machine translation, and natural language inference [2311.09469]. These tasks differ substantially in output space and supervision structure, which the paper uses to support the claim that clarification can be studied in a task-agnostic way [2311.09469].

For the first subtask—determining when clarification is needed—the paper reports that INTENT-SIM consistently outperforms existing uncertainty estimation approaches at identifying predictions that benefit from clarification [2311.09469]. When the system is allowed to ask for clarification on only 10% of examples, it is able to double the performance gains over randomly selecting examples to clarify [2311.09469]. The method is also reported to be robust across a wide range of NLP tasks and language models [2311.09469].

These findings support a more selective view of clarification. The paper does not advocate asking questions whenever ambiguity is possible; instead, it frames the problem as optimizing the utility of interruption. This aligns with later work that treats clarification as an ask-or-answer decision problem under underspecification, including visual question answering settings where clarification is explicitly modeled as a discrete control choice [2601.16400], and dialogue settings that connect clarification frequency to expected regret and action cost [2602.02843].

The paper’s empirical setup also implies a separation between the quality of the clarifying question and the utility of asking it at all. This is important because a system may be capable of generating fluent clarification questions while still failing to identify the inputs for which clarification is worth the interaction cost.

## 5. Relation to later clarification research

Subsequent work extends the clarification paradigm in several directions. In information retrieval, ambiguity-aware prompting methods classify query ambiguity into compact action-oriented types such as Semantic, Generalize, and Specify, using this intermediate structure to guide clarification generation [2504.12113]. This suggests a complementary perspective: whereas INTENT-SIM estimates whether clarification will be useful by modeling entropy over intents, later IR work constrains the reasoning process for what kind of clarification should be asked [2504.12113].

In multimodal settings, later benchmarks show that ambiguity can depend on visual context rather than purely textual underspecification. VAGUE evaluates multimodal intention disambiguation where the intended action is only apparent from the accompanying image, and reports that existing multimodal models remain far below human performance [2411.14137]. Related work on context-dependent VQA treats the correct answer as a function \(a = f(I, q, c)\) of image, question, and missing external context, and uses a Clarify-or-Answer controller to decide whether a clarifying question is needed [2601.16400]. These lines of work expand the original paper’s concern with latent intent into settings where ambiguity is distributed across modalities or external context.

A different extension separates clarification from abstention. Belief-Augmented Generation formulates three strategies—direct answer, clarification question, and abstain—by prompting a model to reason over a sampled belief state, and reports that LLMs by default rarely clarify or abstain even when uncertainty is visible in their sampled outputs [2605.25831]. This suggests that the original “when to clarify” problem may be part of a broader strategic routing problem in which ambiguity-driven clarification and knowledge-driven abstention must be disentangled.

In software and code generation, later systems operationalize clarification before action in highly constrained domains. ProCAD uses a proactive clarifying agent to audit text-to-CAD prompts for ambiguous dimensions, conflicting dimensions, and geometrically impossible dimensions before generating CadQuery programs [2602.03045]. ClarifyCoder similarly trains code LLMs to ask for clarification rather than immediately emit code when requirements are ambiguous, incomplete, or inconsistent [2504.16331]. These systems instantiate the paper’s general thesis in settings where underspecification has direct execution-time consequences.

## 6. Conceptual significance and limitations

The paper’s main significance lies in treating clarification as an explicit computational object rather than an informal conversational nicety. It decomposes the problem into when to clarify, what to ask, and how to use the resulting answer, and it locates the first of these in entropy over latent user intents [2311.09469]. This is conceptually distinct from generic uncertainty estimation, because it isolates the subset of uncertainty that interaction can resolve.

At the same time, the framework depends on simplifying assumptions. The bipartite matching between feasible outputs and clarifying answers gives a clean formalization, but real conversations may involve non-bijective mappings, partial clarifications, or user replies that themselves introduce new ambiguity [2311.09469]. Later multi-turn benchmarks explicitly study such interactional complications, including vague, contradictory, off-focus, and factually wrong user replies, and report that performance degrades as dialogue depth increases [2512.21120]. This suggests that the original framework is strongest as a foundational single-clarification formalization rather than as a full theory of conversational repair.

Another limitation is that the paper’s central contribution is concentrated on the first subtask. The abstract and details emphasize INTENT-SIM and the utility of selective clarification, but provide less comparable emphasis on a unified optimization of question generation and post-clarification response [2311.09469]. A plausible implication is that the framework establishes a decomposition that later work can specialize by domain: ambiguity-aware prompting in IR [2504.12113], reward-based question design in social reasoning [2212.10409], or structured agentic control in multi-turn dialogue [2512.21120].

Despite these limits, the paper lays a foundation for studying clarifying interactions with language models by defining a shared formal vocabulary and by empirically demonstrating that clarification utility can be predicted more effectively when ambiguity is modeled as entropy over user intents rather than as undifferentiated uncertainty [2311.09469].

Source: https://www.emergentmind.com/topics/clarify