Molecular Facts for Fact-Checking

• Molecular facts are rigorously defined, context-sensitive information units that ensure decontextuality and minimality for verifiable claims.
• They are constructed from atomic claims via a two-stage GPT-4 pipeline, balancing specificity with minimal contextual detail for disambiguation.
• Empirical results show that pipelines like MOLECULAR-DECONTEXT reduce ambiguous and over-specified errors, enhancing automated fact-checking accuracy.

Molecular facts are rigorously defined, context-sensitive units of information central to the automated verification of LLM outputs, especially in domains where ambiguity and entity linkage are problematic. In contrast to atomic facts—minimal propositions often stripped of context—molecular facts are constructed to ensure sufficient standalone specificity (decontextuality) without introducing unnecessary detail (minimality). This balances verifiability, non-ambiguity, and evidential generality, supporting accurate and scalable fact-checking in LLM pipelines (Gunjal et al., 2024).

1. Formal Definitions and Desiderata

Molecular facts are defined by two formal criteria: decontextuality and minimality. Let $c_i$ be an atomic claim extracted from an LLM response $r$ to a prompt $x$, such that its truth-conditional interpretation given context $(x, r)$ is $I(c_i \mid x, r)$. A molecular fact $m_i$ is a transformation of $c_i$ that:

• Decontextuality: $I(m_i) = I(c_i \mid x, r)$; the molecule carries the same truth-conditional content as the atomic claim, but interpretable in isolation.
• Minimality: Among all decontextualizations $\mathcal{M}$ of $c_i$, $r$0, where $r$1 is the set of empirical documents that fully support $r$2, and $r$3 is the (oracle) set supporting the intended content.

These criteria jointly ensure that molecular facts are neither so underspecified as to induce ambiguity nor so overspecified as to require rare or unnecessarily detailed evidence.

2. Metrics for Quantification and Evaluation

Operationally, decontextuality is a semantic constraint verified by human or model-based entailment assessments. Minimality is quantified by the size of the evidence set supporting $r$4. Associated metrics include:

• Potential Non-Minimal Rate: The proportion of claims where $r$5 introduces extraneous, unsupported information (e.g., by conflating distinct atomic subclaims).
• Auto Non-Minimal Rate: The frequency with which an automatic fact-checking model changes its label from SUPPORTED (on $r$6) to NOT_SUPPORTED (on $r$7), using a specified evidence paragraph.

For a classifier $r$8 (e.g., RoBERTa-large AlignScore), “auto non-minimality” for a decontextualization $r$9 is defined as: $x$0

This metric directly captures cases where over-specification prevents correct evidence matching.

3. Methodology for Constructing Molecular Facts

A two-stage, GPT-4–based pipeline, termed MOLECULAR-DECONTEXT, systematizes the generation of molecular facts:

Stage 1: Ambiguity Detection

• Extract the main subject $x$1 of $x$2.
• Query whether $x$3 is ambiguous (i.e., shared by multiple real-world entities).
• Recommend a minimal disambiguation attribute $x$4 (such as profession, notable work, or birth year).

Stage 2: Context-Insertion

• Given $x$5, instruct GPT-4 to rewrite $x$6 into $x$7 by explicitly inserting the minimal disambiguating phrase, ensuring all added context is essential for unique reference.

By conditioning on a single disambiguation criterion and tightly bounding the information scope, this approach targets both decontextuality and minimality, avoiding both entity confusion and over-specification.

4. Empirical Comparisons and Benchmarking

Four representative approaches are benchmarked:

Method	Description	Key Results (selected)
ATOMIC	Use $x$8 verbatim	68.7% accuracy (ambiguous entities)
SIMPLE-DECONTEXT	Generic standalone rewrite (GPT-4)	76.2% accuracy, 13.42% auto non-minimal rate, 56% non-minimal human-flagged
SAFE-DECONTEXT	Wei et al. (2024)-style careful rewrite	73.4% accuracy, 3.94% auto non-minimal rate, 43.8% non-minimal human-flagged
MOLECULAR-DECONTEXT	Two-stage molecular pipeline	74.7% accuracy, 52% minimal, 24% non-minimal, 24% still ambiguous (human judgment)

Key findings:

• Over-specification in SIMPLE-DECONTEXT results in error localization failures in up to 13.42% of facts; under-specification (ATOMIC) leads to poor performance (notably for ambiguous entities).
• MOLECULAR-DECONTEXT reduces “wrong-entity” and “multi-evidence” errors while balancing ambiguity and excessive context.

5. Illustrative Examples and Error Modes

Classic failure mode from over-specification:

• “The album was released in 2018.” → SIMPLE-DECONTEXT: “The ‘Blackpink in Your Area’ compilation album was released in 2018.” This merges two subclaims (release year + album type), so a fact-checker may mark it NOT_SUPPORTED if evidence lacks the type annotation.

Molecular fact construction:

• $x$9: “Charles Osgood hosted a radio show.”
• MOLECULAR-DECONTEXT: “Charles Osgood, the American radio broadcaster best known for CBS’s Sunday Morning, hosted a radio show.” This precisely targets the intended entity while omitting surplus features like birth year or education.

6. Implications for Fact Verification Workflows

Results from biography and controlled minimality benchmarks indicate:

• Over-specificity in decontextualizations propagates localization errors (3–10% per multi-claim response).
• Atomic claims are inadequate in ambiguous domains, with accuracy falling below 70% for correctly identifying entity-linked facts.
• The two-stage molecular framework delivers consistent improvements in ambiguous settings, providing minimal and sufficient context for accurate automated and human verification.

The desiderata for molecular facts further suggest their general applicability beyond English biographies, with extensions to other domains, languages, and ambiguity types being promising avenues for future work.

7. Open Directions and Future Applications

Potential developments highlighted:

• Application of molecular fact generation to a wider range of domains (beyond English biographies), including registers and languages with different ambiguity typologies.
• Integration of molecular rewriting into full end-to-end LLM fact-checking pipelines.
• Tuning or distillation of the ambiguity-detection and rewriting modules for efficiency and applicability in open-source or resource-constrained environments.
• Joint optimization of claim decomposition and decontextualization, aiming to minimize error propagation and maximize the coverage of contextual disambiguation.

The molecular fact framework establishes a paradigm for balancing the competing constraints of factual granularity and context, with demonstrable impact on the robustness and accuracy of automated factuality verification systems for LLM outputs (Gunjal et al., 2024).