FactAppeal Framework Overview

• FactAppeal Framework is a comprehensive suite that defines and analyzes factual appeals by modeling epistemic structures, evidentiary bases, and source attributions in text.
• It utilizes annotated datasets, span-level protocols, and advanced models like RoBERTa and Gemma 2 9B to quantify factual appeal identification with high accuracy.
• The framework informs automated fact-checking, media bias analysis, and legal AI by linking claims with external evidentiary sources and epistemic attributes.

The FactAppeal Framework constitutes a suite of approaches, datasets, and computational models dedicated to the identification, segmentation, and structured analysis of factual appeals in text—particularly in news media, judicial reasoning, and automatic fact-checking systems. Unlike conventional frameworks that focus narrowly on factual claim detection or verification, FactAppeal explicitly models the epistemic structures underlying the communication of facts: how claims are anchored by external sources, the granularity of evidence, and the nuanced appeal to authority or experiential rationale. Within this framework, factual statements are not simply verified but are dissected along the dimensions of their evidentiary basis, source attribution, and contextual structure—thus enabling both more rigorous media analysis and the development of explainable, robust fact-checking pipelines.

1. Epistemic Appeal Identification and Factual Structuring

Epistemic Appeal Identification is the foundational task of recognizing whether a factual statement is supported by a reference to external authority, evidence, or source. Within FactAppeal, an epistemic appeal is defined as a factual claim that is explicitly anchored to an external source (Expert, Witness, Official, Direct Evidence, etc.). This distinction is operationalized through span-level annotation protocols that tag textual segments as either “Fact Without Appeal” (asserted on the reporter’s authority alone) or “Fact With Appeal” (supported by a direct or indirect reference). The framework further annotates sources by type, status (named/unnamed), and attribution method (direct quote, indirect quote), and tracks additional context such as source credentials, recipient, temporal and locational markers.

Key attributes of factual appeals include:

Attribute	Description	Example
Source Type	Nature of source (Expert, Witness, etc.)	“Expert Document”
Appeal Attribution	Direct or indirect quotation	Direct quote, paraphrase
Source Naming	Explicit naming or anonymity	“Dr. Wells”, unnamed

This fine-grained epistemic structuring implements a taxonomy that moves beyond surface-level factuality, enabling computational models to distinguish between brute factual assertions and those embedded in sophisticated, evidentiary discourse (Mor-Lan et al., 12 Oct 2025).

2. Datasets, Annotation Protocols, and Model Performance

FactAppeal includes manually annotated resources such as the FactAppeal dataset (3,226 sentences from English-language news) that support rigorous evaluation of epistemic appeal identification. Annotations capture both the factual claims and the supporting appeals, with additional tagging of source roles and contexts. Encoder models (e.g., RoBERTa, DeBERTa v3) and generative decoder models (e.g., Gemma 2 9B) are trained to replicate these annotations. The best generative model achieved a macro-F1 score of 0.73—quantifying its precision and recall in detecting factual appeals and source details.

The annotation protocol, which leverages XML-style embedding of span tags, enables nuanced multi-label, multi-span classification required for probabilistic analysis of appeal structures. This allows for statistical computation of conditional probabilities, such as P(Fact | Sentence) and P(Appeal | Fact), supporting downstream quantitative analyses of reporting practices and epistemic authority distribution (Mor-Lan et al., 12 Oct 2025, Mor-Lan et al., 12 Oct 2025).

3. Computational Analysis of Factual Appeals in Partisan News Media

Applying the FactAppeal framework at scale, recent studies examine epistemic strategies in partisan news outlets (e.g., CNN vs. Fox News) (Mor-Lan et al., 12 Oct 2025). The process involves:

• Segmentation of articles into sentences and annotation for factuality and appeals.
• XML-based tagging of epistemic structure for each claim.
• Quantitative computation of sourcing patterns and conditional appeal probabilities.
• Article matching via title embedding and cosine similarity (Sim(a) = max₍b ∈ Bₜ₎ ( (vₐ ⋅ v_b) / (||vₐ|| ||v_b||) )) to control for topic selection effects.

Findings reveal systemic differences: CNN consistently reports more factual statements and is more likely to support claims with formal appeals to Expert or Expert Document sources, often using indirect quotations with extensive source attribute details. Fox News, in contrast, favors News Report appeals, direct quotes, and only sporadically provides contextual source attributes. Even within matched coverage of the same events, these outlets exhibit sharply divergent epistemic world-construction, demonstrating that credibility is an actively curated property rather than a mere function of reported fact.

4. Integration with Automatic Fact-Checking and Explainability Tools

FactAppeal’s epistemic structuring enhances the transparency and auditability of automated fact-checking systems. By formally linking claims (and their appeals) to external sources and credentials, the framework enables more granular evaluation of explanation actionability, evidentiary sufficiency, and correction efficacy. For instance, evaluation paradigms such as FinGrAct (Eldifrawi et al., 7 Apr 2025) and FaStFact (Wan et al., 13 Oct 2025) incorporate claim segmentation, error detection, correction mapping, and verification against aggregated evidence. Precision in source attribution (name, credential, type) and segmentation of appeals contributes directly to the robustness and interpretability of these systems, as evidenced by increased correlation with human judgment and reduced bias in automated evaluation (Eldifrawi et al., 7 Apr 2025).

5. Applications in Judicial Reasoning and LegalAI

Expanding beyond media analysis, FactAppeal principles inform schema development for legal reasoning, particularly in appellate scenarios and structured adjudication. The AppealCase dataset (Huang et al., 22 May 2025) employs multi-dimensional annotation of paired trial and appeal documents, focusing on judgment reversals, reversal reasons (factual or legal), cited legal provisions, and claim-level evolution. Tree-structured representations (e.g., Factum Probandum hierarchy (Shen et al., 2 Mar 2025)) incorporate evidence tracking and experiential reasoning nodes, illustrating how appeals to authority and explicit evidentiary chains structure final verdict outcomes. In such contexts, FactAppeal’s emphasis on explicit appeal linkage becomes foundational for both transparency and explainability in AI-assisted law.

6. Research Implications, Limitations, and Future Directions

The FactAppeal Framework advances epistemic analysis by decoupling claim detection from the deeper identification of evidentiary and authority appeals. Its dataset and protocol innovations facilitate quantitative media analysis, robust fact-checking, and the construction of modular pipelines for context-sensitive credibility assessment. Challenges remain around multi-source appeals (especially nested appeals), scaling model capacity for longer texts, handling ambiguous appeal attribution, and generalizing across languages and domains. Continued research will investigate more expressive modeling of epistemic networks, fine-grained attribution disambiguation, and integration of multimodal evidence.

A plausible implication is that as epistemic structures become more central to public trust in information systems, FactAppeal-inspired frameworks will be pivotal for automated, explainable, and critically robust fact analysis in both media and legal contexts.