Claim-Conditioned Probability (CCP)

• Claim-Conditioned Probability (CCP) is a framework that generalizes traditional conditional probability through conditional expectations defined on plausible preorders over algebraic structures.
• It employs algebraic and measure-free constructions to extend inference to cases with zero-probability events and supports robust probabilistic reasoning.
• CCP is applied in neural language models to quantify token-level uncertainty, demonstrating improved performance in detecting hallucinations compared to standard metrics.

Claim-Conditioned Probability (CCP) is a formal framework designed to generalize the notion of conditional probability to settings where classical constructions break down, including cases involving conditioning on claims or events of probability zero and contexts such as uncertainty quantification in neural text generation. This framework arises both as a mathematical foundation for conditional inference and as a practical tool for evaluating the epistemic confidence of models or knowledge-based systems under various forms of evidence and logical structure.

1. Mathematical Foundations of Conditioning and Preorders

The core of CCP in recent mathematical treatments is the reinterpretation of probability as a special case of conditional expectation induced by a plausible preorder on a commutative real algebra of random quantities. A random quantity is an element of a commutative real algebra $T$; events are idempotent elements $A\in T$ with $A\cdot A=A$, forming a Boolean algebra under the operations:

• Conjunction: $A\wedge B := A\cdot B$
• Negation: $\neg A := 1-A$
• Disjunction: $A\vee B := A+B-A\cdot B$

A plausible preorder $\leq$ on $T$ satisfies:

1. Plausibility: If $A$ is an event, $0\leq A$.
2. Additivity: If $A\in T$0 and $A\in T$1, then $A\in T$2.
3. Homogeneity: If $A\in T$3 and $A\in T$4 (real), then $A\in T$5.
4. Extension: $A\in T$6 iff $A\in T$7.

For $A\in T$8, the expectation induced by $A\in T$9 is $A\cdot A=A$0, with well-defined behavior for infinite or coincident extrema. Conditional expectation is defined for any event $A\cdot A=A$1 by $A\cdot A=A$2, and $A\cdot A=A$3 is then the expectation under this conditional preorder.

A partial function $A\cdot A=A$4 is called coherent if and only if it arises as the restriction of some $A\cdot A=A$5 for a plausible preorder, yielding a general characterization theorem for conditional probabilities that includes degenerate cases $A\cdot A=A$6 (Mečíř, 2019).

2. Coherent Conditional Probability and Claims

Applying this machinery, for any event $A\cdot A=A$7 and "claim" $A\cdot A=A$8 (not necessarily with $A\cdot A=A$9), define the claim-conditioned probability as

$A\wedge B := A\cdot B$0

with $A\wedge B := A\cdot B$1 as above. Properties include:

• $A\wedge B := A\cdot B$2 for any $A\wedge B := A\cdot B$3
• Monotonicity: if $A\wedge B := A\cdot B$4, $A\wedge B := A\cdot B$5
• Subadditivity and Bayes' rule hold in full generality
• $A\wedge B := A\cdot B$6 if $A\wedge B := A\cdot B$7 and $A\wedge B := A\cdot B$8 are disjoint; $A\wedge B := A\cdot B$9 if $\neg A := 1-A$0 implies $\neg A := 1-A$1
• Definition extends to $\neg A := 1-A$2 with $\neg A := 1-A$3 or $\neg A := 1-A$4 undefined

This construction provides a rigorous and order-theoretically sound basis for conditional reasoning not limited by the usual restrictions of the Kolmogorov framework (Mečíř, 2019).

3. Measure-Free Algebraic Constructions

Independently, a measure-free approach to conditional probability constructs an extended algebra of events and "conditional objects" $\neg A := 1-A$5 (for $\neg A := 1-A$6 events in a Boolean algebra $\neg A := 1-A$7), defined as principal-ideal cosets:

$\neg A := 1-A$8

These new objects admit a semantics in which any (finitely additive) $\neg A := 1-A$9 consistently extends to $A\vee B := A+B-A\cdot B$0 when $A\vee B := A+B-A\cdot B$1. The construction is minimal and necessary for consistently interpreting inference rules as conditional statements in probabilistic expert systems (Goodman, 2013).

Key algebraic properties include:

• Identities
  • $A\vee B := A+B-A\cdot B$2
  • $A\vee B := A+B-A\cdot B$3
  • $A\vee B := A+B-A\cdot B$4
• Bayes’ rule and chaining law for conditional objects
• Computations on conditional objects are polynomial in the size of the Boolean representation of $A\vee B := A+B-A\cdot B$5

This algebra provides a syntax and calculus for conditional reasoning, supporting applications in knowledge-based inference, evidence fusion, and sequential updating (Goodman, 2013).

4. Claim-Conditioned Probability in LLMs

Recently, CCP has been operationalized as a metric of token-level uncertainty in autoregressive neural LLMs, with direct application to hallucination detection and model calibration (Fadeeva et al., 2024). For a set of tokens associated with an atomic claim, CCP is defined as follows.

At each position $A\vee B := A+B-A\cdot B$6, let $A\vee B := A+B-A\cdot B$7 be the top-$A\vee B := A+B-A\cdot B$8 candidate tokens given context $A\vee B := A+B-A\cdot B$9. For each $\leq$0, a pretrained NLI (Natural Language Inference) model determines the logical relation (entailment, contradiction, neutral) with the reference output $\leq$1, distinguishing between shared claim-type and factual value.

The token-level CCP is:

$\leq$2

where the numerator sums probabilities over alternatives entailing $\leq$3 (same claim value), and the denominator over alternatives with the same claim type (could be contradictory values). This design factors out uncertainty about claim-type and surface form, isolating factual uncertainty.

Claim-level CCP aggregates as

$\leq$4

A high $\leq$5 implicates high uncertainty and probable hallucination.

5. Empirical Evaluation and Implementation Notes

The CCP metric, as applied to LLM outputs, systematically outperforms baselines such as maximum token probability, entropy, and "ask-the-model" scores for factuality discrimination. Experimental setups have used various open and proprietary models (Vicuna 13b, Mistral 7b, Jais 13b, GPT-3.5, GPT-4, Yi 6B), with automatic and human annotation of claims in biography-generation tasks across multiple languages.

Quantitative results on ROC-AUC for discriminating supported vs. false claims show consistent superiority of CCP by $\leq$6–$\leq$7 points over baselines. Example cases demonstrate that CCP successfully isolates uncertainty localized at factual content, while other scores respond to irrelevant surface variability or claim-type uncertainty. Implementation involves top-$\leq$8 beam search, efficient NLI evaluation, and computational tractability with $\leq$9–$T$0; product-based aggregation at claim level maximizes detection accuracy (Fadeeva et al., 2024).

6. Practical Applications and Theoretical Significance

CCP establishes a uniform formalism that:

• Admits order-theoretic, algebraic (measure-free), and neural uncertainty quantification instantiations
• Generalizes conditional probability for inference rules, evidence combination, and hypothesis updating even when conditioning on zero-probability or undefined events
• Provides rigorous, computable frameworks for expert systems and for white-box hallucination detection in generative neural models

Since CCP is underpinned by a minimal extension of classical structures (plausible preorders, coset algebras), its integrity and computational feasibility are maintained across logic-based and data-driven systems (Mečíř, 2019, Goodman, 2013, Fadeeva et al., 2024).

7. Relation to Broader Frameworks and Limitations

The CCP methodology unifies aspects of conditional probability under a common umbrella, admitting both order-theoretic and algebraic perspectives, and linking formal epistemic reasoning with statistical and neural machine learning contexts. It circumvents traditional singularities (e.g., $T$1) and supports generalization of inference rules. However, the operational definition of claims, and the practical implementation of NLI-based CCP in language generation, depend on system-specific architectures and assumptions about token-claim alignment and NLI reliability.

A plausible implication is that CCP provides a stable substrate for future research on fact-checking, uncertainty quantification, and reasoning under both symbolic and statistical paradigms. Limitations may arise insofar as the quality of the underlying models (e.g., NLI, atomization of claims) influences CCP’s discriminative power in empirical contexts (Mečíř, 2019, Fadeeva et al., 2024, Goodman, 2013).