Labelled Uncertainty Graph (LUG)

• Labelled Uncertainty Graph (LUG) is a graph-based framework that compactly represents uncertainty through labels on nodes, applied in both belief-space planning and long-form LLM outputs.
• In belief-space planning, LUG extends classical planning graphs using labeled BDDs to estimate distances amidst multiple possible worlds and derive heuristics like h_max, h_sum, and h_level.
• For long-form LLM outputs, LUG models sampled responses and atomic claims in a bipartite graph, utilizing centrality metrics for claim-level uncertainty estimation and uncertainty-aware decoding.

Searching arXiv for the specified LUG papers and closely related context. arXiv_search(query="(Jiang et al., 2024)", max_results=5) arXiv_search(query="Graph-based Uncertainty Metrics for Long-form LLM Outputs", max_results=5) arxiv_search("Graph-based Uncertainty Metrics for Long-form LLM Outputs", 5) search_arxiv(query="Graph-based Uncertainty Metrics for Long-form LLM Outputs", max_results=5) functions.arxiv_search({"query":"Labelled Uncertainty Graph", "max_results": 10}) “Labelled Uncertainty Graph” (LUG) denotes two distinct graph-based formalisms in AI. In belief-space planning, the LUG is an extension of the classical planning-graph structure in which each node at level $k$ carries a label $\ell_k(x)$ over world atoms, enabling distance estimation between belief states and the extraction of heuristics such as $h_{\max}$, $h_{\sum}$, $h_{level}$, and $h_{RP}$ (Bryce et al., 2011). In recent work on long-form LLM outputs, the Labelled Uncertainty Graph represents LLM generations and the atomic claims they entail as a bipartite graph, and uses graph-centrality metrics as surrogate confidence labels for claim-level uncertainty estimation and uncertainty-aware decoding (Jiang et al., 2024). The shared acronym reflects a common emphasis on compactly encoding uncertainty in graph structure, but the two uses operate in different problem settings and with different semantics.

1. Terminological scope and conceptual role

In the planning literature, the LUG is defined over a belief state $BS_p$ represented as a propositional formula over a finite set of fluents $F$, with $Worlds(BS_p)=\{S_1,\ldots,S_n\}$ the possible worlds that satisfy that formula (Bryce et al., 2011). Its purpose is to provide a single labelled graph that can emulate what would otherwise require exponentially many classical planning graphs, while still supporting informed estimates of the cost to reach a belief-state goal under uncertainty.

In the long-form LLM setting, the Labelled Uncertainty Graph is introduced for “Claim-Level Confidence Estimation and Uncertainty-Aware Decoding” (Jiang et al., 2024). Here the graph is bipartite, with one partition for sampled responses and one for deduplicated atomic claims. The graph is used to assign fine-grained, claim-level confidence scores without model internals, and then to guide a two-stage decoding procedure—filter followed by synthesis—to trade off factuality and informativeness.

A common misconception is to treat “LUG” as a single standardized object across AI subfields. The published definitions do not support that interpretation. The two formalisms share the idea of labelled graph structure under uncertainty, but one is a planning-graph data structure for belief-space search, whereas the other is a graph-based uncertainty framework for long-form LLM generations [(Bryce et al., 2011); (Jiang et al., 2024)].

2. LUG for claim-level confidence estimation in long-form LLM outputs

The long-form LLM LUG begins from the observation that modern black-box LLMs are prone to hallucination, particularly in long-form outputs that mix correct and incorrect claims (Jiang et al., 2024). The framework therefore represents the relationship between LLM generations and claims within them as a bipartite graph and estimates the claim-level uncertainty with a family of graph centrality metrics.

The construction procedure is explicit. Given a user prompt $x$, response sampling first produces a greedy response $\ell_k(x)$0 and $\ell_k(x)$1 additional samples $\ell_k(x)$2, with $\ell_k(x)$3, optionally more up to $\ell_k(x)$4 or $\ell_k(x)$5. Claim decomposition and de-duplication then prompt the LLM to decompose each sampled response $\ell_k(x)$6 in a subset $\ell_k(x)$7 into atomic claims $\ell_k(x)$8. To remove semantically redundant claims, the method iteratively merges sets, with $\ell_k(x)$9 and, for $h_{\max}$0, $h_{\max}$1, where $h_{\max}$2 is implemented by prompting the LLM to test entailment between every candidate in $h_{\max}$3 and the accumulated set $h_{\max}$4 and appending only novel claims. The final claim node set is $h_{\max}$5.

Edge establishment builds a bipartite graph $h_{\max}$6. For every pair $h_{\max}$7, the system prompts “Does $h_{\max}$8 entail $h_{\max}$9? Yes/No.” If yes, it adds undirected edge $h_{\sum}$0 to $h_{\sum}$1. Post-processing may remove any claim nodes that never appear in $h_{\sum}$2, and the graph is stored as adjacency matrix $h_{\sum}$3 with nodes ordered $h_{\sum}$4 (Jiang et al., 2024).

The associated pseudocode, $h_{\sum}$5, returns claim set $h_{\sum}$6 and uncertainties $h_{\sum}$7. Its steps are: sample responses; decompose claims for $h_{\sum}$8; merge all claim sets; test entailment for each $h_{\sum}$9; build $h_{level}$0 and $h_{level}$1; compute centralities such as $h_{level}$2 and $h_{level}$3 for each claim $h_{level}$4; and set $h_{level}$5 for closeness centrality (Jiang et al., 2024).

3. Centrality, self-consistency, and uncertainty-aware decoding

Once the LLM LUG is built, each claim node $h_{level}$6 is assigned a confidence score equal to its chosen centrality $h_{level}$7, with high centrality interpreted as low uncertainty and uncertainty defined as $h_{level}$8 (Jiang et al., 2024). This formulation gives a graph-theoretic reinterpretation of self-consistency: under this view, existing uncertainty estimation methods based on the concept of self-consistency can be viewed as using degree centrality as an uncertainty measure.

The primary centrality measures described are degree centrality and closeness centrality. Degree centrality is

$h_{level}$9

with normalized form

$h_{RP}$0

Closeness centrality uses shortest-path distance in the graph. Let $h_{RP}$1 be the shortest-path distance in $h_{RP}$2. Using the Wasserman–Faust correction for disconnected graphs,

$h_{RP}$3

where $h_{RP}$4 is the connected component containing $h_{RP}$5 (Jiang et al., 2024).

The decoding procedure consumes $h_{RP}$6 in four steps. First, select threshold $h_{RP}$7 such as the $h_{RP}$8-th percentile of $h_{RP}$9 on a small dev set. Second, define operational claims $BS_p$0. Third, issue an overview prompt, “Integrate these facts into a coherent paragraph:” followed by the list $BS_p$1. Fourth, decode $BS_p$2. Varying $BS_p$3 traces out a precision–informativeness curve: tighter $BS_p$4 higher factuality, fewer claims (Jiang et al., 2024).

A second misconception is that the framework is only a relabeling of standard self-consistency. The reported formulation does not support that reduction. Degree centrality recovers standard self-consistency, while higher-order metrics such as closeness exploit the global graph structure for more accurate uncertainty estimation (Jiang et al., 2024).

4. Empirical behavior in long-form generation

The experimental setup for the LLM LUG uses the datasets FActScore, PopQA-Long, and Natural Questions, with models GPT-3.5-turbo, GPT-4, and Llama-3-70B (Jiang et al., 2024). The uncertainty baselines are IL-VC, PH-VC, P(True), SC, and SC + VC. Evaluation for uncertainty estimation uses AUROC (True vs. False) and AUPRC-Negative (focus on false claims).

The main findings are quantitative. For $BS_p$5 or $BS_p$6, SC outperforms VC baselines by approximately $BS_p$7–$BS_p$8 AUPRC. Closeness centrality $BS_p$9 further improves AUPRC by $F$0 on average $F$1, and AUROC by up to $F$2. Betweenness, PageRank, and eigenvector centrality yield modest gains over SC but below $F$3 (Jiang et al., 2024). In the excerpted Table 1, for GPT-4 with $F$4, AUPRC-Negative is $F$5 for IL-VC, $F$6 for PH-VC, $F$7 for SC, and $F$8 for $F$9; for GPT-4 with $Worlds(BS_p)=\{S_1,\ldots,S_n\}$0, the corresponding values are $Worlds(BS_p)=\{S_1,\ldots,S_n\}$1, $Worlds(BS_p)=\{S_1,\ldots,S_n\}$2, $Worlds(BS_p)=\{S_1,\ldots,S_n\}$3, and $Worlds(BS_p)=\{S_1,\ldots,S_n\}$4.

For decoding, greedy and CoVe are dominated by all UAD variants. UAD($Worlds(BS_p)=\{S_1,\ldots,S_n\}$5) achieves a Pareto-optimal frontier: at $Worlds(BS_p)=\{S_1,\ldots,S_n\}$6 precision, it includes $Worlds(BS_p)=\{S_1,\ldots,S_n\}$7 more true claims vs. SC. Across the full curve, UAD($Worlds(BS_p)=\{S_1,\ldots,S_n\}$8) yields $Worlds(BS_p)=\{S_1,\ldots,S_n\}$9–$x$0 absolute factuality at matched informativeness (Jiang et al., 2024).

The ablation results explain why closeness centrality is discriminative. False claims have larger average shortest-path distances to other claims than true claims. Performance also steadily rises with more samples when varying $x$1, confirming that centrality benefits from richer graph connectivity. The limitations are equally explicit: graph construction increases inference latency because it requires $x$2 decomposition plus $x$3 entailment calls; claim decomposition assumes claims are context-independent and atomic; and future directions include lightweight claim clustering, end-to-end prompts that jointly extract, score, and synthesize claims in a single pass, and extensions to non-bipartite graphs for structured claims such as event chains (Jiang et al., 2024). This suggests that the main trade-off is not only estimation quality but also the cost of constructing the graph.

5. LUG in belief-space planning: formal structure and label propagation

In belief-space planning, the LUG is an extension of the classical planning-graph structure, with levels $x$4, each comprising three layers: literal layer $x$5, action layer $x$6, and effect layer $x$7 (Bryce et al., 2011). Each node $x$8 at level $x$9 carries a label $\ell_k(x)$00, which is a propositional formula over a fresh set of world atoms $\ell_k(x)$01, one $\ell_k(x)$02 per world $\ell_k(x)$03. Intuitively, $\ell_k(x)$04 is true under $\ell_k(x)$05 iff $\ell_k(x)$06 is reachable in the projection of the classical planning graph from world $\ell_k(x)$07 in $\ell_k(x)$08 steps or fewer.

The initial belief state is represented as a BDD over $\ell_k(x)$09 encoding $\ell_k(x)$10, constrained so that exactly those $\ell_k(x)$11 are true that correspond to actual worlds of $\ell_k(x)$12. At the initial layer $\ell_k(x)$13, for each fluent literal $\ell_k(x)$14,

$\ell_k(x)$15

that is, the BDD that is the disjunction of those $\ell_k(x)$16 whose state $\ell_k(x)$17 assigns $\ell_k(x)$18 to true. At the action layer, for each action $\ell_k(x)$19 whose execution-precondition formula $\ell_k(x)$20 is reachable at level $\ell_k(x)$21,

$\ell_k(x)$22

and the action is included if $\ell_k(x)$23. At the effect layer, for each conditional effect $\ell_k(x)$24,

$\ell_k(x)$25

and the effect is included if the label is not $\ell_k(x)$26. The next literal layer is then

$\ell_k(x)$27

The operator $\ell_k(x)$28 extends labels from literals to arbitrary formulas:

$\ell_k(x)$29

$\ell_k(x)$30

and $\ell_k(x)$31 (Bryce et al., 2011).

Construction continues until the pair $\ell_k(x)$32 stabilizes or until the goal belief-formula $\ell_k(x)$33 is reachable, that is, $\ell_k(x)$34. The representation uses BDDs throughout: one BDD variable $\ell_k(x)$35 for each possible world, the initial belief $\ell_k(x)$36, and subsequent labels computed by BDD operations AND, OR, and NOT. Internally, the CUDD package, or any efficient BDD library, maintains a shared BDD for all labels, exploiting common subgraphs (Bryce et al., 2011).

6. Planning heuristics, guarantees, and the CBTC example

Once the planning LUG is built to level $\ell_k(x)$37 where $\ell_k(x)$38 first becomes reachable, several heuristic-distance estimates can be read off (Bryce et al., 2011). If the goal $\ell_k(x)$39 is put into CNF $\ell_k(x)$40, then for each clause

$\ell_k(x)$41

The heuristic

$\ell_k(x)$42

is admissible because every clause must be supported in parallel. The heuristic

$\ell_k(x)$43

is generally inadmissible because separate clause support might share the same actions. The level heuristic is

$\ell_k(x)$44

and is also admissible. Relaxed-plan extraction yields $\ell_k(x)$45 by choosing effects whose labels cover all worlds for each clause, projecting back to actions and their preconditions, and summing the number of selected actions across layers:

$\ell_k(x)$46

This is not guaranteed admissible but is usually far more informed (Bryce et al., 2011).

The key guarantees are stated directly. $\ell_k(x)$47 and $\ell_k(x)$48 are admissible lower-bounds on the true distance from $\ell_k(x)$49 to $\ell_k(x)$50. $\ell_k(x)$51 and $\ell_k(x)$52 are generally inadmissible since they may double-count shared actions. For consistency, $\ell_k(x)$53 and $\ell_k(x)$54 satisfy the triangle-inequality style requirement for A*: if $\ell_k(x)$55, then one can show from their definitions that $\ell_k(x)$56. Empirically, $\ell_k(x)$57, and $\ell_k(x)$58 drastically reduces search expansions, at the cost of higher per-node compute (Bryce et al., 2011).

The “Courteous BTC” example gives a worked instance. The fluents are $\ell_k(x)$59. The initial $\ell_k(x)$60 says $\ell_k(x)$61, so $\ell_k(x)$62 where $\ell_k(x)$63 and $\ell_k(x)$64. The goal is $\ell_k(x)$65. Introduce BDD vars $\ell_k(x)$66, so $\ell_k(x)$67 is $\ell_k(x)$68. At level $\ell_k(x)$69,

$\ell_k(x)$70

$\ell_k(x)$71

Flush has precondition $\ell_k(x)$72 and effect $\ell_k(x)$73 unconditionally, so $\ell_k(x)$74. At $\ell_k(x)$75, $\ell_k(x)$76 appears at $\ell_k(x)$77 with label $\ell_k(x)$78. DunkP1 and DunkP2 then become executable at $\ell_k(x)$79, and their conditional effects for $\ell_k(x)$80 get labels $\ell_k(x)$81 and $\ell_k(x)$82, respectively, so at $\ell_k(x)$83 one gets $\ell_k(x)$84 labelled $\ell_k(x)$85. The graph stops at $\ell_k(x)$86 because $\ell_k(x)$87 is reachable there. The heuristic values are

$\ell_k(x)$88

$\ell_k(x)$89

Relaxed-plan extraction yields $\ell_k(x)$90. In this small problem, a strong plan requires three sequential actions, so $\ell_k(x)$91 underestimates the serial cost, while $\ell_k(x)$92 and $\ell_k(x)$93 match it exactly (Bryce et al., 2011).

The planning usage also clarifies a third misconception: not all LUG-based heuristics are admissible. The formal statements distinguish admissible lower-bounds such as $\ell_k(x)$94 and $\ell_k(x)$95 from generally inadmissible but more informed heuristics such as $\ell_k(x)$96 and $\ell_k(x)$97 (Bryce et al., 2011).