Papers
Topics
Authors
Recent
Search
2000 character limit reached

Uncertainty Externalization Overview

Updated 4 July 2026
  • Uncertainty externalization is the process of transferring latent risks, assumptions, and value judgments from core analysis to an external, explicit interface.
  • In energy policy modelling, this approach critiques false certainty by highlighting hidden assumptions, while in LLMs it improves calibration and error control.
  • Across domains like MBSE, agent systems, and FX market making, externalization serves to either obscure or operationalize uncertainty, balancing risk and actionable insights.

Searching arXiv for the cited papers and topic-specific references to ground the article. arxiv_search(query="\"Unpacking uncertainty in the modelling process for energy policy making\" OR \"LLMs Should Express Uncertainty Explicitly\" OR \"Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertainty\" OR \"Confidence Laundering in Agent Systems\" OR \"Not All Uncertainty Is Equal\" OR \"Uncertainty Modeling for SysML v2\" OR \"Fighting Uncertainty with Uncertainty\" OR \"Market making by an FX dealer\"", max_results=10) Uncertainty externalization denotes a family of operations in which uncertainty is displaced from one locus of analysis or control to another rather than eliminated. In energy-policy modelling, it refers to pushing assumptions, value judgments, and gaps in knowledge outside the formal analysis, thereby creating an “excess of certainty” around particular policy options (Piano et al., 2021). In recent LLM work, the same expression is used in an almost opposite sense: latent uncertainty is made explicit as a trainable interface for abstention, retrieval, verification, and intervention (Guo et al., 7 Apr 2026). Related literatures use the term for preserving uncertainty across agent handoffs through latent carriers (Shi et al., 9 Jun 2026), encoding uncertainty as first-class structure in SysML v2 (Zhang et al., 25 Feb 2026), transferring inventory risk from an FX dealer’s book to the inter-dealer market (Barzykin et al., 2021), or introducing randomness into the decision rule itself through “Randoptimization” (Kashyap, 2016). Across these uses, the central question is not whether uncertainty exists, but whether it is hidden, exposed, propagated, or operationalized at the decision interface.

1. Domain-specific meanings

The term has no single universal definition across arXiv literatures. Instead, it names several distinct interface phenomena.

Domain Meaning of uncertainty externalization Representative paper
Energy policy modelling Uncertainty is pushed outside formal analysis (Piano et al., 2021)
LLM inference Uncertainty is expressed explicitly for control (Guo et al., 7 Apr 2026)
LLM reasoning Latent uncertainty is verbalized as epistemic information (Kim et al., 16 Mar 2026)
Agent systems Fragility is preserved across handoffs via latent carriers (Shi et al., 9 Jun 2026)
MBSE / SysML v2 Uncertainty is modeled explicitly in system structure (Zhang et al., 25 Feb 2026)
Decision theory / operations Randomness is introduced into the policy itself (Kashyap, 2016)
FX market making Inventory risk is transferred to the market (Barzykin et al., 2021)

In the energy-modelling literature, externalization is a pathology: “black-box” models, single cost-optimal pathways, and reliance on a single model or indicator suppress deep uncertainty and close off alternatives (Piano et al., 2021). In LLM interface design, by contrast, externalization is treated as a design goal: uncertainty should be a first-class, trainable interface that the model exposes explicitly to downstream controllers (Guo et al., 7 Apr 2026). In agent systems, the emphasis shifts from local estimation to uncertainty-preserving interface design, because uncertainty propagates only when it survives the handoff between components (Shi et al., 9 Jun 2026).

This suggests that the common denominator is representational location. What changes across fields is whether moving uncertainty outward is regarded as loss, preservation, or deliberate control.

2. Energy-policy modelling and the critique of false certainty

In “Unpacking uncertainty in the modelling process for energy policy making” (Piano et al., 2021), uncertainty externalization is defined as the tendency of energy-system modelling to generate crisp scenarios or single “best-estimate” indicators while leaving unexamined assumptions, institutional choices, contested framings, and non-quantifiable uncertainties outside the formal analysis. The paper locates this problem in Post-Normal Science, where facts are uncertain, stakes are high, values are disputed and decisions are urgent. Externalization manifests as “black-box” models, fixation on one or two cost-optimal pathways, neglect of qualitative or “deep” uncertainties, and reliance on a single model or indicator for major policy commitments.

The proposed remedies are three reflexive-modelling methodologies. NUSAP decomposes a quantity into Numeral, Unit, Spread, Assessment, and Pedigree. Its pedigree component scores each assumption against proxy representation, empirical basis, methodological rigour, validation, and theoretical understanding. In matrix form, if PP is the pedigree matrix, an aggregate score may be written as

pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.

Diagnostic diagrams then place each assumption on a two-axis scatter plot with pedigree on the horizontal axis and sensitivity on the vertical axis. The “Danger zone” is Q4Q4: high sensitivity and poor pedigree. Sensitivity auditing extends beyond technical UA & SA to interrogate framing, scope-definition, rhetoric of numbers, hidden value judgments, and governance assumptions. Its mathematical hook is the first-order Sobol’ index

Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).

The four detailed case studies show how these methods recapture uncertainty that conventional modelling had externalized. In the ExternE case, a transdisciplinary workshop identified 30 assumptions, reduced them to six critical ones, and found that all six received weak aggregate pedigree. In ESME, BioRES and CCSmbr lay in Q4Q4, implying high influence on total system cost but very weak pedigree. In the NETs-in-IAMs case, Vaughan & Gough’s 18-expert workshop found that almost all NETs parameters sat in the Q4Q4 danger zone. In the Ecological Footprint case, the seven-point auditing checklist exposed omitted sink functions, hidden technical coefficients, absent error bars, and no published uncertainty or sensitivity analysis.

The significance of this literature is methodological rather than merely computational. It treats uncertainty as partly epistemic, value-laden, and institutional, so quality assurance requires workshops, pedigree matrices, diagnostic plots, and sensitivity auditing rather than only tighter confidence intervals.

3. Explicit uncertainty in LLMs: global confidence and local signals

“LLMs Should Express Uncertainty Explicitly” reframes uncertainty as an interface for control rather than a latent quantity estimated post hoc (Guo et al., 7 Apr 2026). It distinguishes two decision problems. Outcome uncertainty—“Should I trust this final answer?”—is addressed by a global scalar. Process uncertainty—“Am I hitting a fragile step in my chain-of-thought?”—is addressed by local signals during generation. The global interface appends a decimal confidence token c∈[0,1]c\in[0,1] after generating a reasoning trajectory z1:Tz_{1:T} and an answer y^\hat y, with

c≈Pr⁡(Y=1∣h1:T),Y=1{y^=gold}.c \approx \Pr(Y=1\mid h_{1:T}),\qquad Y=\mathbf1\{\hat y=\text{gold}\}.

The local interface allows the model to emit the literal token “<uncertain>” during reasoning, with

pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.0

Both interfaces are learned through a light post-training stage using a variant of GRPO. The idealized update is

pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.1

For verbalized confidence,

pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.2

which rewards justified confidence and penalizes over-confident errors. For “<uncertain>”, the reward order is

pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.3

implemented as pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.4.

The reported effects are large. On 2WikiMultihopQA, verbalized confidence changes accuracy from pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.5, ECE from pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.6, Brier from pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.7, NLL from pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.8, and the over-confidence gap from pedigreeScorei=∑j=15wjPij.\mathrm{pedigreeScore}_i=\sum_{j=1}^5 w_j P_{ij}.9; wrong answers with Q4Q40 drop from Q4Q41 to Q4Q42 (Guo et al., 7 Apr 2026). In Adaptive RAG aggregated over five datasets, Verbal-Calibrate reaches EM/F1 Q4Q43 at Q4Q44 retrieval, while Uncertain-Calibrate reaches Q4Q45 at Q4Q46 retrieval. Mechanistically, the verbal interface concentrates token-level KL divergence almost exclusively at the final confidence digit and preserves layer geometry, whereas the special-token interface induces progressive late-layer CKA drop to Q4Q47, indicating a broader reorganization.

A complementary theoretical account is given in “Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertainty” (Kim et al., 16 Mar 2026). That paper decomposes reasoning into procedural information and epistemic verbalization. If Q4Q48 is a latent internal variable, then making it verbalized and therefore conditionable yields

Q4Q49

The paper further argues that purely procedural reasoning can become informationally stagnant, whereas sporadic epistemic verbalization can ensure continued information acquisition. Empirically, masking epistemic tokens in top-performers drops accuracy by Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).0 and Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).1; removing all epistemic verbalization from high-quality procedural traces degrades Qwen3-14B base from Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).2 pass@1 on AIME24.

A frequent misconception is that uncertainty expression is merely a stylistic overlay. These two papers argue the opposite: explicit uncertainty can alter calibration, retrieval control, and the internal routing of computation.

4. Propagation across interfaces: agent handoffs and model-based systems engineering

In multi-step agent systems, the central problem is not only whether uncertainty exists locally, but whether it is preserved at component boundaries. “Confidence Laundering in Agent Systems: Why Uncertainty Needs a Latent Carrier” defines uncertain decision handoff as the transfer of an intermediate decision made under uncertainty (Shi et al., 9 Jun 2026). If Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).3 is an upstream distribution over possible artifacts and the interface applies a deterministic commitment map

Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).4

then distinct epistemic states can collapse to the same Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).5. Downstream components receive a procedurally valid artifact but lose access to its fragility. That failure mode is called confidence laundering. The proposed remedy is a latent uncertainty carrier Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).6, attached to the committed artifact through an augmented handoff

Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).7

In HotpotQA web-search experiments, latent carriers achieve AUROC Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).8 in binary uncertainty detection, versus Si=VarXi[E(Y∣Xi)]/Var(Y).S_i=\mathrm{Var}_{X_i}[E(Y\mid X_i)]/\mathrm{Var}(Y).9 with scalar scores and Q4Q40 with answer-only.

A structurally analogous agenda appears in “Uncertainty Modeling for SysML v2” (Zhang et al., 25 Feb 2026). There, uncertainty is externalized into explicit model constructs by incorporating the PSUM metamodel into SysML v2. The core stereotypes include Q4Q41, Q4Q42, Q4Q43, Q4Q44, Q4Q45, and Q4Q46. Uncertainty carries attributes such as Q4Q47, Q4Q48, Q4Q49, and Q4Q40. Propagation is represented by explicit references from uncertainties to effects, so downstream consequences remain traceable rather than implicit. The extension was validated on seven publicly available SysML v2 models totaling 3,685 lines of model and over 1,000 elements.

The common significance is interface preservation. In agent systems, preservation requires a latent carrier attached to decision handoffs. In MBSE, preservation requires stereotypes, constraints, measurements, and effect links. In both cases, uncertainty becomes actionable only when it is retained in a form that downstream processes can consume.

5. Externalization as controlled transfer: randomized policies and hedging

A different tradition treats uncertainty externalization as a deliberate control strategy. In “Fighting Uncertainty with Uncertainty: A Baby Step,” Kashyap calls this “Randoptimization” (Kashyap, 2016). The idea is to choose a randomized policy rather than a single deterministic action, thereby pushing some randomness into the decision-maker’s own control. In the benchmark news-vendor problem, deterministic profit under order quantity Q4Q41 is

Q4Q42

with classical optimizer

Q4Q43

Randoptimization instead draws Q4Q44, yielding

Q4Q45

A sufficient condition for randomized performance to match or exceed the deterministic optimum is

Q4Q46

Under the simplifying constraint Q4Q47, Kashyap gives an additional bound on how spread-out Q4Q48 may be. The paper sketches applications to school admissions, job-candidate selection, journal submissions, stock picking, and monetary policy.

In FX market making, externalization has a more literal market meaning. “Market making by an FX dealer: tiers, pricing ladders and hedging rates for optimal risk control” defines internalization as absorbing client flow imbalances on-book and using price-skewing to attract offsetting trades, whereas externalization means trading in the inter-dealer segment to immediately hedge inventory (Barzykin et al., 2021). The dealer controls bid/ask skews and an external execution rate Q4Q49. The reference price follows

c∈[0,1]c\in[0,1]0

With execution cost c∈[0,1]c\in[0,1]1, the optimal hedging rate is

c∈[0,1]c\in[0,1]2

where c∈[0,1]c\in[0,1]3. If c∈[0,1]c\in[0,1]4, then c∈[0,1]c\in[0,1]5: pure internalization. For a realistic EURUSD top-tier bank, the paper reports c∈[0,1]c\in[0,1]6 bps/c∈[0,1]c\in[0,1]7, c∈[0,1]c\in[0,1]8 dayc∈[0,1]c\in[0,1]9, z1:Tz_{1:T}0 bps·day per M€, z1:Tz_{1:T}1 bps, z1:Tz_{1:T}2 bps per M€, and z1:Tz_{1:T}3 bpsz1:Tz_{1:T}4 per M€; under these parameters, the dealer fully internalizes for z1:Tz_{1:T}5–z1:Tz_{1:T}6 M€ and internalizes about z1:Tz_{1:T}7 of flow.

These two literatures use the same word for distinct mechanisms. Randoptimization externalizes environmental uncertainty into a controlled sampling distribution. FX hedging externalizes inventory risk to the broader market. In both cases, uncertainty is not removed; it is transferred into a mechanism with explicit design parameters.

6. Granularity, verification, and the limits of visibility

Making uncertainty visible does not guarantee that downstream actors will use it well. “Not All Uncertainty Is Equal: How Uncertainty Granularity Shapes Human Verification in LLM-Assisted Decision Making” studies output-level, relation-level, and token-level uncertainty displays in a between-subjects study with z1:Tz_{1:T}8 (Villavicencio et al., 27 May 2026). Token-level uncertainty is computed from next-token distributions, for example through per-token entropy

z1:Tz_{1:T}9

or through y^\hat y0, where y^\hat y1. Relation-level uncertainty assigns a score y^\hat y2 or y^\hat y3–y^\hat y4 to each reasoning step or claim. Output-level uncertainty provides a single scalar y^\hat y5 for the entire response.

The behavioral effects differ sharply by granularity. Agreement with AI is Baseline y^\hat y6, UQ-Token y^\hat y7, UQ-Relation y^\hat y8, and UQ-Output y^\hat y9; UQ-Token exceeds Baseline with c≈Pr⁡(Y=1∣h1:T),Y=1{y^=gold}.c \approx \Pr(Y=1\mid h_{1:T}),\qquad Y=\mathbf1\{\hat y=\text{gold}\}.0. Confidence in one’s own answers decreases under UQ-Output and UQ-Relation but not under UQ-Token. Relation-level uncertainty reduces external verification: link clicks fall from Baseline c≈Pr⁡(Y=1∣h1:T),Y=1{y^=gold}.c \approx \Pr(Y=1\mid h_{1:T}),\qquad Y=\mathbf1\{\hat y=\text{gold}\}.1 to c≈Pr⁡(Y=1∣h1:T),Y=1{y^=gold}.c \approx \Pr(Y=1\mid h_{1:T}),\qquad Y=\mathbf1\{\hat y=\text{gold}\}.2 with c≈Pr⁡(Y=1∣h1:T),Y=1{y^=gold}.c \approx \Pr(Y=1\mid h_{1:T}),\qquad Y=\mathbf1\{\hat y=\text{gold}\}.3, and web searches fall from c≈Pr⁡(Y=1∣h1:T),Y=1{y^=gold}.c \approx \Pr(Y=1\mid h_{1:T}),\qquad Y=\mathbf1\{\hat y=\text{gold}\}.4 to c≈Pr⁡(Y=1∣h1:T),Y=1{y^=gold}.c \approx \Pr(Y=1\mid h_{1:T}),\qquad Y=\mathbf1\{\hat y=\text{gold}\}.5 with c≈Pr⁡(Y=1∣h1:T),Y=1{y^=gold}.c \approx \Pr(Y=1\mid h_{1:T}),\qquad Y=\mathbf1\{\hat y=\text{gold}\}.6. Task accuracy and completion time show no reliable differences.

The result is a direct caution against an intuitive but overgeneral claim: more detailed uncertainty cues are not necessarily more responsible. Token-level heatmaps can increase agreement with the AI, whereas relation-level cues can steer users away from independent fact-checking. This suggests that uncertainty externalization must be evaluated not only by calibration metrics or representational fidelity, but also by the behavioral pathways it induces in verification, oversight, and intervention.

Across these literatures, the decisive issue is therefore not visibility alone. The consequential properties are whether uncertainty is retained at the right interface, encoded at the right granularity, and matched to an action that can actually use it.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Uncertainty Externalization.