Epistemic Appeal Identification

• Epistemic appeal identification is the process of isolating and characterizing methods that lend credibility to factual assertions through explicit links to evidence or experts.
• It employs multi-layered modeling, using token-level classification and generative approaches to differentiate direct from indirect appeals with high annotation reliability.
• The approach supports practical applications in fact-checking, media bias analysis, and discourse parsing by quantifying how authority and trust are constructed in texts.

Epistemic appeal identification refers to the systematic process of isolating and characterizing the ways in which factual assertions are endowed with credibility via reference to external sources or evidence. Distinguished from claim detection or classical verification, it explicitly focuses on how claims are anchored to structures—such as expert testimony, documents, or direct evidence—that enhance their epistemic status. This notion is especially pertinent in domains like news media, where epistemic appeals function as the mechanisms that facilitate trust, persuasion, and the construction of a shared reality among audiences (Mor-Lan et al., 12 Oct 2025).

1. Task Definition and Scope

Epistemic appeal identification is operationalized as a multi-layered annotation and modeling challenge: it requires not only segmenting factual from non-factual language, but also determining the type, source, and evidentiary character of appeals that underpin those claims. Unlike standard verification pipelines (which ask whether a claim is supported by any evidence), epistemic appeal identification targets the granularity of how the claim’s credibility is constructed—discriminating, for example, between direct evidence (such as a photograph), citation of an expert, or invocation of an official (Mor-Lan et al., 12 Oct 2025).

The FactAppeal dataset, developed specifically for this task, contains 3,226 English-language news sentences manually annotated at the span level with both factual and appeal properties. It supports nuanced differentiation along the following axes:

• “Fact Without Appeal”: Pure factual assertion, with no epistemic support cited.
• “Fact With Appeal”: Factual statement accompanied by explicit reference to sources or evidence.
• Appeal type: Method of appeal (direct/indirect quotation, paraphrase) and presence/absence of named source.
• Source attributes: Role, credentials, recipient, time, and location.

Annotation reliability is high, with Intersection over Union 0.74 and Cohen’s Kappa 0.82 (Mor-Lan et al., 12 Oct 2025).

2. Typology of Epistemic Appeals and Sources

The paper introduces a fine-grained typology for epistemic appeals that distinguishes sources both in terms of their proximity to events and the nature of their authority:

• Internal/human evidence: Active Participants (directly involved), Witnesses (observers), Officials (authority figures with direct or bureaucratic connection), Direct Evidence (non-human, e.g., photographs).
• External/human and documentary evidence: Expert (specialist knowledge), Expert Document (scholarly papers/reports), News Report (third-party journalistic citation).

Appeals may be either direct (verbatim quotation) or indirect (paraphrase, summary). The framework supports identification not only of what kind of entity is cited, but of how the citation is linguistically encoded and whether additional epistemic credentials (such as a title or expertise claim) are explicitly mentioned.

3. Modeling Approaches and Evaluation

Epistemic appeal identification employs both token-level encoder models (e.g., RoBERTa, DeBERTa v3, ModernBERT) and generative decoder architectures (Gemma 2 in 2B–9B parameter ranges, Llama, Mistral).

• Token-level approaches: Treat the problem as multi-label, multi-span classification at the word level, allowing for overlapping or nested tag assignments (factuality, appeal attribute) per token.
• Decoder-based approaches: Task the model with producing an XML-style marked-up version of the source sentence, where factual and appeal features are annotated as tags.

Evaluation employs macro-averaged word-level precision, recall, and F1 across all tag categories (18 in total), with best performance from generative models in the 9B parameter range (macro-F1 = 0.73) (Mor-Lan et al., 12 Oct 2025).

The use of focal loss (Lin et al., 2018) addresses class imbalance: $L = -\alpha \, (1-p_t)^\gamma\, \log(p_t)$ where $p_t$ is the model’s estimated probability for the true class.

4. Epistemic Structures and Their Annotation

A key innovation of the FactAppeal resource is its explicit modeling of epistemic structure at the span level. Each annotation encodes:

• The presence or absence of a factual claim.
• The linkage of the claim to a source (spanning a direct quotation, source mention, or paraphrased assertion).
• The Mapping of source type to a typology (as above), with further attributes such as namedness (is the source explicitly identified), role/credential, and method of attribution (e.g., indirect speech).
• Supplementary information, including to whom the appeal is directed (Recipient), and when/where it is made.

Such annotations permit the downstream quantification and analysis of different patterns of epistemic appeal, enabling rigorous comparison across datasets or outlets.

5. Applications in Automated Assessment and Media Analysis

Identifying epistemic appeals provides a crucial foundation for downstream applications:

• Fact-checking and verification: Appeals to experts, documents, or evidence provide additional context for automated systems that must assess claim reliability.
• Media studies and bias analysis: Patterns of appeal can be correlated with differences in news outlet sourcing strategy, offering formal underpinnings for the paper of epistemic bias and the construction of credibility (Mor-Lan et al., 12 Oct 2025).
• Communication and social epistemology: Mapping epistemic appeal types at scale allows for the exploration of the “social construction” of facts—how communities or institutions build trust and accept shared truths.
• Advanced discourse parsing: Embedding epistemic appeal structures enables richer models of argument structure, authority, and persuasion than those restricted to surface-level claim verification.

6. Significance, Limitations, and Implications

Epistemic appeal identification formalizes the bridge between facticity and credibility. By distinguishing the backbone by which news and other texts attempt to legitimate their claims, it exposes the epistemic mechanisms that underlie influence and trust in public discourse.

Limitations include dependence on annotated resources, the challenge of distinguishing nuanced forms of attribution (e.g., sarcasm, implied authority), and the need to generalize beyond the news domain. However, the methodology, supported by robust annotation agreement and scalable modeling, signals an important advance in the automated analysis of epistemic structure.

In conclusion, this concept enables computational systems to critically parse not just whether a statement is a factual claim, but how, why, and by what means that fact is to be believed—a core object of paper for research at the intersection of language, reasoning, and epistemology.