---
title: Labelled Uncertainty Graph (LUG)
url: https://www.emergentmind.com/topics/labelled-uncertainty-graph-lug
type: topic
---

# Labelled Uncertainty Graph (LUG)

Searching arXiv for the specified LUG papers and closely related context.
arXiv_search(query="2410.20783", max_results=5)
arXiv_search(query="Graph-based Uncertainty Metrics for Long-form Language Model Outputs", max_results=5)
arxiv_search("Graph-based Uncertainty Metrics for Long-form Language Model Outputs", 5)
search_arxiv(query="Graph-based Uncertainty Metrics for Long-form Language Model Outputs", max_results=5)
functions.arxiv_search({"query":"Labelled Uncertainty Graph", "max_results": 10})
“Labelled Uncertainty Graph” (LUG) denotes two distinct graph-based formalisms in AI. In belief-space planning, the LUG is an extension of the classical planning-graph structure in which each node at level $k$ carries a label $\ell_k(x)$ over world atoms, enabling distance estimation between belief states and the extraction of heuristics such as $h_{\max}$, $h_{\sum}$, $h_{level}$, and $h_{RP}$ [1103.1711]. In recent work on long-form language model outputs, the Labelled Uncertainty Graph represents LLM generations and the atomic claims they entail as a bipartite graph, and uses graph-centrality metrics as surrogate confidence labels for claim-level uncertainty estimation and uncertainty-aware decoding [2410.20783]. The shared acronym reflects a common emphasis on compactly encoding uncertainty in graph structure, but the two uses operate in different problem settings and with different semantics.

## 1. Terminological scope and conceptual role

In the planning literature, the LUG is defined over a belief state $BS_p$ represented as a propositional formula over a finite set of fluents $F$, with $Worlds(BS_p)=\{S_1,\ldots,S_n\}$ the possible worlds that satisfy that formula [1103.1711]. Its purpose is to provide a single labelled graph that can emulate what would otherwise require exponentially many classical planning graphs, while still supporting informed estimates of the cost to reach a belief-state goal under uncertainty.

In the long-form LLM setting, the Labelled Uncertainty Graph is introduced for “Claim-Level Confidence Estimation and Uncertainty-Aware Decoding” [2410.20783]. Here the graph is bipartite, with one partition for sampled responses and one for deduplicated atomic claims. The graph is used to assign fine-grained, claim-level confidence scores without model internals, and then to guide a two-stage decoding procedure—filter followed by synthesis—to trade off factuality and informativeness.

A common misconception is to treat “LUG” as a single standardized object across AI subfields. The published definitions do not support that interpretation. The two formalisms share the idea of labelled graph structure under uncertainty, but one is a planning-graph data structure for belief-space search, whereas the other is a graph-based uncertainty framework for long-form LLM generations [1103.1711; 2410.20783].

## 2. LUG for claim-level confidence estimation in long-form LLM outputs

The long-form LLM LUG begins from the observation that modern black-box LLMs are prone to hallucination, particularly in long-form outputs that mix correct and incorrect claims [2410.20783]. The framework therefore represents the relationship between LLM generations and claims within them as a bipartite graph and estimates the claim-level uncertainty with a family of graph centrality metrics.

The construction procedure is explicit. Given a user prompt $x$, response sampling first produces a greedy response $r_0 \leftarrow M_{T=0}(x)$ and $N-1$ additional samples $r_1 \ldots r_{N-1} \leftarrow M_{T=t>0}(x)$, with $R \leftarrow \{r_0,r_1,\ldots,r_{N-1}$, optionally more up to $|R|=5$ or $10\}$. Claim decomposition and de-duplication then prompt the LLM to decompose each sampled response $r_i$ in a subset $R_N \subseteq R$ into atomic claims $C_i=\{c_{i,1},\ldots,c_{i,k}\}$. To remove semantically redundant claims, the method iteratively merges sets, with $C^{(1)} \leftarrow C_1$ and, for $i=2 \ldots N$, $C^{(i)} \leftarrow Merge(C^{(i-1)},C_i)$, where $Merge(\cdot,\cdot)$ is implemented by prompting the LLM to test entailment between every candidate in $C_i$ and the accumulated set $C^{(i-1)}$ and appending only novel claims. The final claim node set is $C \leftarrow C^{(N)}$.

Edge establishment builds a bipartite graph $G=(R \cup C,E)$. For every pair $(r,c)\in R \times C$, the system prompts “Does $r$ entail $c$? Yes/No.” If yes, it adds undirected edge $(r,c)$ to $E$. Post-processing may remove any claim nodes that never appear in $r_0$, and the graph is stored as adjacency matrix $A$ with nodes ordered $[R;C]$ [2410.20783].

The associated pseudocode, $BuildLUG(x,N,|R|,t)$, returns claim set $C$ and uncertainties $U(c)$. Its steps are: sample responses; decompose claims for $i=1 \ldots N$; merge all claim sets; test entailment for each $r \in R, c \in C$; build $G$ and $A$; compute centralities such as $C_D(v)$ and $C_C(v)$ for each claim $v \in C$; and set $U(v)\leftarrow -C_C(v)$ for closeness centrality [2410.20783].

## 3. Centrality, self-consistency, and uncertainty-aware decoding

Once the LLM LUG is built, each claim node $v \in C$ is assigned a confidence score equal to its chosen centrality $C_*(v)$, with high centrality interpreted as low uncertainty and uncertainty defined as $U(v)=-C_*(v)$ [2410.20783]. This formulation gives a graph-theoretic reinterpretation of self-consistency: under this view, existing uncertainty estimation methods based on the concept of self-consistency can be viewed as using degree centrality as an uncertainty measure.

The primary centrality measures described are degree centrality and closeness centrality. Degree centrality is
$$
C_D(v)=deg(v)=\sum_{u\in R} A_{vu},
$$
with normalized form
$$
C_D(v)=\frac{deg(v)}{|R|}.
$$
Closeness centrality uses shortest-path distance in the graph. Let $d(v,u)$ be the shortest-path distance in $G$. Using the Wasserman–Faust correction for disconnected graphs,
$$
C_C(v)=\frac{|V|-1}{\sum_{u\neq v} d(v,u)} \cdot \frac{|V|}{|V_v|},
$$
where $V_v$ is the connected component containing $v$ [2410.20783].

The decoding procedure consumes $(C,U)$ in four steps. First, select threshold $\delta$ such as the $q$-th percentile of $U$ on a small dev set. Second, define operational claims $C^o \leftarrow \{c \in C \mid U(c)<\delta\}$. Third, issue a synthesis prompt, “Integrate these facts into a coherent paragraph:” followed by the list $C^o$. Fourth, decode $y^* \leftarrow M_0(prompt)$. Varying $\delta$ traces out a precision–informativeness curve: tighter $\delta \rightarrow$ higher factuality, fewer claims [2410.20783].

A second misconception is that the framework is only a relabeling of standard self-consistency. The reported formulation does not support that reduction. Degree centrality recovers standard self-consistency, while higher-order metrics such as closeness exploit the global graph structure for more accurate uncertainty estimation [2410.20783].

## 4. Empirical behavior in long-form generation

The experimental setup for the LLM LUG uses the datasets FActScore, PopQA-Long, and Natural Questions, with models GPT-3.5-turbo, GPT-4, and Llama-3-70B [2410.20783]. The uncertainty baselines are IL-VC, PH-VC, P(True), SC, and SC + VC. Evaluation for uncertainty estimation uses AUROC (True vs. False) and AUPRC-Negative (focus on false claims).

The main findings are quantitative. For $|R|=5$ or $10$, SC outperforms VC baselines by approximately $10$–$15\%$ AUPRC. Closeness centrality $C_C$ further improves AUPRC by $+6.8\%$ on average $(p<0.05)$, and AUROC by up to $+6\%$. Betweenness, PageRank, and eigenvector centrality yield modest gains over SC but below $C_C$ [2410.20783]. In the excerpted Table 1, for GPT-4 with $|R|=5$, AUPRC-Negative is $0.394$ for IL-VC, $0.531$ for PH-VC, $0.651$ for SC, and $0.733$ for $C_C$; for GPT-4 with $|R|=10$, the corresponding values are $0.394$, $0.531$, $0.708$, and $0.751$.

For decoding, greedy and CoVe are dominated by all UAD variants. UAD($C_C$) achieves a Pareto-optimal frontier: at $95\%$ precision, it includes $70\%$ more true claims vs. SC. Across the full curve, UAD($C_C$) yields $+2$–$4\%$ absolute factuality at matched informativeness [2410.20783].

The ablation results explain why closeness centrality is discriminative. False claims have larger average shortest-path distances to other claims than true claims. Performance also steadily rises with more samples when varying $N$, confirming that centrality benefits from richer graph connectivity. The limitations are equally explicit: graph construction increases inference latency because it requires $N \times$ decomposition plus $|R|\times|C|$ entailment calls; claim decomposition assumes claims are context-independent and atomic; and future directions include lightweight claim clustering, end-to-end prompts that jointly extract, score, and synthesize claims in a single pass, and extensions to non-bipartite graphs for structured claims such as event chains [2410.20783]. This suggests that the main trade-off is not only estimation quality but also the cost of constructing the graph.

## 5. LUG in belief-space planning: formal structure and label propagation

In belief-space planning, the LUG is an extension of the classical planning-graph structure, with levels $k=0,1,2,\ldots$, each comprising three layers: literal layer $\mathcal{L}_k$, action layer $\mathcal{A}_k$, and effect layer $\mathcal{E}_k$ [1103.1711]. Each node $x$ at level $k$ carries a label $\ell_k(x)$, which is a propositional formula over a fresh set of world atoms $W=\{w_1,\ldots,w_n\}$, one $w_i$ per world $S_i$. Intuitively, $\ell_k(x)$ is true under $w_i$ iff $x$ is reachable in the projection of the classical planning graph from world $S_i$ in $k$ steps or fewer.

The initial belief state is represented as a BDD over $W$ encoding $\bigvee_i w_i$, constrained so that exactly those $w_i$ are true that correspond to actual worlds of $BS_p$. At the initial layer $k=0$, for each fluent literal $\ell \in F \cup \neg F$,
$$
\ell_0(\ell) \leftarrow BDD(BS_p \wedge \text{“}\ell \text{ holds in world } w_i\text{”}),
$$
that is, the BDD that is the disjunction of those $w_i$ whose state $S_i$ assigns $\ell$ to true. At the action layer, for each action $a$ whose execution-precondition formula $\rho^e(a)$ is reachable at level $k$,
$$
\ell_k(a)\leftarrow \ell_k^*(\rho^e(a)),
$$
and the action is included if $\ell_k(a)\neq \bot$. At the effect layer, for each conditional effect $\phi^j(a):\rho^j(a)\Rightarrow \epsilon^j(a)$,
$$
\ell_k(\phi^j(a)) \leftarrow \ell_k(a)\wedge \ell_k^*(\rho^j(a)),
$$
and the effect is included if the label is not $\bot$. The next literal layer is then
$$
\ell_{k+1}(\ell)\leftarrow \bigvee_{\phi^j(a)\in \mathcal{E}_k:\,\ell\in \epsilon^j(a)} \ell_k(\phi^j(a)).
$$

The operator $\ell_k^*$ extends labels from literals to arbitrary formulas:
$$
\ell_k^*(f\wedge g)=\ell_k^*(f)\wedge \ell_k^*(g), \quad
\ell_k^*(f\vee g)=\ell_k^*(f)\vee \ell_k^*(g),
$$
$$
\ell_k^*(\neg f)=\neg \ell_k^*(f), \quad
\ell_k^*(\top)=BS_p,\quad \ell_k^*(\bot)=\bot,
$$
and $\ell_k^*(literal)=\ell_k(literal)$ [1103.1711].

Construction continues until the pair $(all\ \ell_k(\ell),\ell_{k+1}(\ell))$ stabilizes or until the goal belief-formula $BS_i$ is reachable, that is, $BS_p \models \ell_k^*(BS_i)$. The representation uses BDDs throughout: one BDD variable $w_i$ for each possible world, the initial belief $BS_p(w)=\bigvee_{S_i \models BS_p} w_i$, and subsequent labels computed by BDD operations AND, OR, and NOT. Internally, the CUDD package, or any efficient BDD library, maintains a shared BDD for all labels, exploiting common subgraphs [1103.1711].

## 6. Planning heuristics, guarantees, and the CBTC example

Once the planning LUG is built to level $B$ where $BS_i$ first becomes reachable, several heuristic-distance estimates can be read off [1103.1711]. If the goal $BS_i$ is put into CNF $\kappa(BS_i)=\bigwedge_{C\in Clauses} C$, then for each clause
$$
level(C)=\min\{k \mid BS_P\_BDD \models LABEL\_EVAL(C,\ell\_label[k])\}.
$$
The heuristic
$$
h_{\max}(BS_i)=\max_{C\in \kappa(BS_i)} level(C)
$$
is admissible because every clause must be supported in parallel. The heuristic
$$
h_{\sum}(BS_i)=\sum_{C\in \kappa(BS_i)} level(C)
$$
is generally inadmissible because separate clause support might share the same actions. The level heuristic is
$$
h_{level}(BS_i)=\min\{k \mid BS_P\_BDD \models LABEL\_EVAL(BS_i,\ell\_label[k]) \text{ and no intra-clause literals are mutex}\},
$$
and is also admissible. Relaxed-plan extraction yields $h_{RP}(BS_i)$ by choosing effects whose labels cover all worlds for each clause, projecting back to actions and their preconditions, and summing the number of selected actions across layers:
$$
h_{RP}(BS_i)=\sum_{r=0}^{B-1} |\mathcal{A}_r|.
$$
This is not guaranteed admissible but is usually far more informed [1103.1711].

The key guarantees are stated directly. $h_{\max}$ and $h_{level}$ are admissible lower-bounds on the true distance from $BS_p$ to $BS_i$. $h_{\sum}$ and $h_{RP}$ are generally inadmissible since they may double-count shared actions. For consistency, $h_{\max}$ and $h_{level}$ satisfy the triangle-inequality style requirement for A*: if $BS' = Regress(BS,a)$, then one can show from their definitions that $h(BS') \leq h(BS)+1$. Empirically, $h_{RP} \geq h_{level} \geq h_{\max}$, and $h_{RP}$ drastically reduces search expansions, at the cost of higher per-node compute [1103.1711].

The “Courteous BTC” example gives a worked instance. The fluents are $\{arm, clog, inP1, inP2\}$. The initial $BS_p$ says $arm \wedge clog \wedge (inP1 \oplus inP2)$, so $Worlds(BS_p)=\{S_1,S_2\}$ where $S_1=(arm,clog,inP1,\neg inP2)$ and $S_2=(arm,clog,\neg inP1,inP2)$. The goal is $BS_i:\neg arm \wedge \neg clog$. Introduce BDD vars $w_1,w_2$, so $BS_p$ is $w_1 \vee w_2$. At level $0$,
$$
\ell_0(inP1)=w_1,\quad \ell_0(\neg inP2)=w_1,\quad \ell_0(inP2)=w_2,\quad \ell_0(\neg inP1)=w_2,
$$
$$
\ell_0(clog)=w_1\vee w_2,\quad \ell_0(arm)=w_1\vee w_2.
$$
Flush has precondition $\top$ and effect $\neg clog$ unconditionally, so $\ell_0(Flush)=w_1\vee w_2$. At $k=0$, $\neg clog$ appears at $\mathcal{L}_1$ with label $w_1\vee w_2$. DunkP1 and DunkP2 then become executable at $k=1$, and their conditional effects for $\neg arm$ get labels $w_1$ and $w_2$, respectively, so at $\mathcal{L}_2$ one gets $\neg arm$ labelled $w_1\vee w_2$. The graph stops at $B=2$ because $BS_i$ is reachable there. The heuristic values are
$$
h_{\max}=\max(level(\neg arm), level(\neg clog))=\max(2,1)=2,
$$
$$
h_{\sum}=2+1=3.
$$
Relaxed-plan extraction yields $h_{RP}=3$. In this small problem, a strong plan requires three sequential actions, so $h_{\max}$ underestimates the serial cost, while $h_{RP}=3$ and $h_{\sum}=3$ match it exactly [1103.1711].

The planning usage also clarifies a third misconception: not all LUG-based heuristics are admissible. The formal statements distinguish admissible lower-bounds such as $h_{\max}$ and $h_{level}$ from generally inadmissible but more informed heuristics such as $h_{\sum}$ and $h_{RP}$ [1103.1711].

Source: https://www.emergentmind.com/topics/labelled-uncertainty-graph-lug