Reliability Estimation Branch

• Reliability Estimation Branch is a component that assigns continuous reliability scores to news sources via hyperlink propagation on a weighted, directed graph.
• It employs forward and reverse reinforcement learning formulations and investment propagation methods, achieving up to macro F1 scores around 81% in evaluation.
• The framework is scalable, language-independent, and integrates seamlessly with downstream verification pipelines for tasks like fake-news detection and source ranking.

To satisfy the requirement that all claims trace to the provided data, the article will focus on the “Reliability Estimation Branch” as defined in (Burdisso et al., 2024). It will avoid using the 2026 valence–arousal paper because the data block explicitly states that the supplied content is not the actual paper and instead provides a generic specification. It will also avoid introducing cross-domain generalizations unless clearly marked as interpretation. Citations will primarily use (Burdisso et al., 2024), with no unsupported claims. A Reliability Estimation Branch is a component that assigns a continuous reliability degree to each news source by leveraging how sources interact on the web, rather than predicting discrete labels from content. In the formulation introduced in "Reliability Estimation of News Media Sources: Birds of a Feather Flock Together" (Burdisso et al., 2024), the branch operates on a weighted directed source graph induced by hyperlinks, propagates positive and negative reliability signals through that graph, and outputs a scalar score $\rho(s) \in \mathbb{R}$ for every source $s$. Larger positive values indicate stronger evidence of reliability, while larger negative values indicate stronger unreliability; a zero threshold can be used when categorical labels are required (Burdisso et al., 2024).

1. Problem formulation and conceptual basis

The branch is defined over a large set $S$ of news media websites and seeks a function $\rho : S \to \mathbb{R}$ such that $\rho(s) > 0$ if $s$ is reliable, $\rho(s) \le 0$ if $s$ is unreliable, and $\rho(s) < \rho(s')$ if $s'$ is more reliable than $s$0 (Burdisso et al., 2024). The central design choice is to estimate a reliability degree rather than a reliability label. This degree-based formulation supports ranking and prioritization in downstream tasks better than categorical labels, allows correlation analysis with journalist-provided scores, and enables threshold tuning and calibration across domains (Burdisso et al., 2024).

The operational principle is homophily on the hyperlink network, summarized by the phrase “Birds of a Feather Flock Together” (Burdisso et al., 2024). Frequent linking is treated as endorsement or citation behavior and as a high-probability navigation path for a random reader. Hyperlinks are described as a proxy for content-based relationships while remaining scalable and language-independent. Reliability then propagates along interaction edges: sources that are strongly connected to reliable outlets accrue positive signals, whereas sources that channel readers to unreliable outlets accrue negative signals (Burdisso et al., 2024).

This branch is deliberately content- and language-independent. It relies solely on the hyperlink network and does not use GNNs or centrality features in training, although PageRank is used as a baseline for correlation analysis. Content-based features from prior work are used only for ensemble comparisons (Burdisso et al., 2024).

2. Source graph construction and supervision

The underlying graph is a weighted, directed graph $s$1, where an edge $s$2 exists if source $s$3 contains articles that hyperlink to source $s$4, and the edge weight $s$5 is the proportion of total outbound hyperlinks in $s$6 that point to $s$7 (Burdisso et al., 2024). Formally, each directed edge has weight

$s$8

where $s$9 aggregates all hyperlinks from all articles in source $S$0 (Burdisso et al., 2024).

The graph is built from Common Crawl News (CC-News) snapshots using a Python pipeline that parses WARC files to extract article URLs and hyperlinks, filters by language with fastText, and aggregates links at the domain level (Burdisso et al., 2024). Focusing on English, the graph combines four August snapshots from 2019, 2020, 2021, and 2022, totaling roughly 103M articles and yielding 17,057 sources and 909,354 directed edges (Burdisso et al., 2024).

Ground truth is aggregated from four sources. MBFC contributes 4,138 labels after merging factuality ratings into reliability classes; Wikipedia’s Perennial Sources contributes 553 labels mapped to reliable, mixed, and unreliable; a curated fake news domain list contributes 556 unreliable sources; and NewsGuard contributes 85 journalist-provided scores in $S$1 based on nine transparent criteria (Burdisso et al., 2024). The aggregated labeled dataset contains 5,332 sources, and evaluation uses only domains present in the graph (Burdisso et al., 2024).

Three evaluation sets are defined by different treatments of the mixed class. ExpsetA contains 294 reliable and 106 unreliable sources, ExpsetB contains 1,630 reliable and 487 unreliable sources, and ExpsetB$S$2 contains 1,630 reliable and 166 unreliable sources (Burdisso et al., 2024). The data block explicitly notes that the graph and labeled set are an order of magnitude larger than datasets commonly used for source profiling (Burdisso et al., 2024).

3. Propagation mechanisms and core equations

The branch uses multiple propagation strategies. The principal reinforcement learning formulation casts reliability estimation as a Markov Decision Process $S$3 on the source graph (Burdisso et al., 2024). States are the news domains, there is a single action that moves a virtual user from the current source to a neighbor, and the transition probability is the normalized hyperlink strength:

$S$4

Rewards are assigned from ground-truth labels with

$S$5

where $S$6 for reliable sources, $S$7 for unreliable sources, and $S$8 otherwise (Burdisso et al., 2024).

The value-based forward formulation defines F-Reliability through the Bellman equation

$S$9

with $\rho : S \to \mathbb{R}$0 (Burdisso et al., 2024). Because there is only one action, the policy is implicit and $\rho : S \to \mathbb{R}$1 (Burdisso et al., 2024). This is a “future-looking” degree that estimates how much a source guides users toward reliable or unreliable destinations (Burdisso et al., 2024).

The reverse formulation defines P-Reliability using

$\rho : S \to \mathbb{R}$2

with $\rho : S \to \mathbb{R}$3 (Burdisso et al., 2024). This is a “past-looking” degree in which sources accrue reliability as more reliable sources lead to them, or unreliability if they are mainly reached from unreliable sources (Burdisso et al., 2024).

To model asymmetry, the paper defines FP-Reliability as

$\rho : S \to \mathbb{R}$4

where $\rho : S \to \mathbb{R}$5 is the forward value using only negative rewards and $\rho : S \to \mathbb{R}$6 is the reverse value using only positive rewards (Burdisso et al., 2024). This explicitly separates penalization for pointing to unreliable outlets from reward for being cited by reliable outlets (Burdisso et al., 2024).

In parallel with these RL formulations, the branch includes an investment-style propagation method denoted I-Reliability. It initializes $\rho : S \to \mathbb{R}$7 and iterates three steps (Burdisso et al., 2024):

$\rho : S \to \mathbb{R}$8

$\rho : S \to \mathbb{R}$9

$\rho(s) > 0$0

This update is repeated $\rho(s) > 0$1 times, aggregating influence from up to $\rho(s) > 0$2-hop neighborhoods (Burdisso et al., 2024).

The implementation uses deterministic value iteration or deterministic investment updates. There is no gradient-based learning, and no Q-learning, actor-critic, or policy-gradient machinery (Burdisso et al., 2024). Value iteration initializes $\rho(s) > 0$3 for all nodes and repeats Bellman updates until the maximum absolute update $\rho(s) > 0$4 becomes small enough (Burdisso et al., 2024).

4. From reliability degrees to labels and measured performance

The branch outputs a continuous reliability degree for every node in the graph (Burdisso et al., 2024). When downstream categorical labels are required, the decision rule is a threshold at zero: reliable if $\rho(s) > 0$5, unreliable if $\rho(s) > 0$6 (Burdisso et al., 2024). The paper states that practical confidence can be derived from the margin $\rho(s) > 0$7 or by ensembling runs across folds or snapshots, because no separate uncertainty estimator is provided (Burdisso et al., 2024).

Hyperparameters are tuned through 5-fold cross-validation on the reliability classification task. For the RL variants, the discount factor is selected from $\rho(s) > 0$8; for I-Reliability, the number of iterations is selected from $\rho(s) > 0$9 (Burdisso et al., 2024). Best settings generally favor local neighborhoods, with $s$0 and $s$1, which indicates that nearby interactions carry most of the signal (Burdisso et al., 2024).

The classification protocol applies rewards only to training folds, computes $s$2 for all 17k nodes, and assigns held-out labels using the zero threshold (Burdisso et al., 2024). Macro-averaged $s$3 is reported together with class-wise $s$4. The paper gives

$s$5

On ExpsetB$s$6, I-Reliability achieves macro-averaged $s$7, with reliable-class $s$8 and unreliable-class $s$9 (Burdisso et al., 2024).

For ordinal validation, the branch is compared to NewsGuard journalist scores on 85 domains. Spearman’s rank correlation is computed as

$\rho(s) \le 0$0

where $\rho(s) \le 0$1 is the rank difference for source $\rho(s) \le 0$2 (Burdisso et al., 2024). With rewards available, P-Reliability reaches Spearman $\rho(s) \le 0$3; without rewards for those 85 domains, it remains strong at $\rho(s) \le 0$4; PageRank reaches $\rho(s) \le 0$5 (Burdisso et al., 2024). The paper states that these results confirm that the continuous $\rho(s) \le 0$6 captures ordinal reliability and aligns with journalist assessments (Burdisso et al., 2024).

Ablation results also shape the branch design. Larger graphs obtained by joining snapshots improve performance and reduce uncertainty. Small $\rho(s) \le 0$7 and $\rho(s) \le 0$8 perform best, emphasizing local interactions. FP-Reliability underperforms for binary classification because of asymmetries in forward and backward normalization, but contributes positively to rank correlation and can be averaged with P-Reliability to strengthen ordinal alignment (Burdisso et al., 2024).

5. Integration pattern and operational role

Within a larger verification pipeline, the branch is described as an integration module with clearly specified inputs, processing stages, and outputs (Burdisso et al., 2024). Its inputs are a source graph $\rho(s) \le 0$9 built from a web crawl or ingestion pipeline and seed rewards $s$0 for a subset of sources, obtained from internal curation, MBFC or Wikipedia lists, paid rating services, or prior pipelines (Burdisso et al., 2024).

The recommended processing choices are P-Reliability or I-Reliability (Burdisso et al., 2024). The branch computes continuous $s$1 for all sources, can optionally normalize $s$2 to $s$3, and returns both the degree and an optional label via the threshold rule (Burdisso et al., 2024). Confidence can be exposed via $s$4 or ensemble variance (Burdisso et al., 2024).

The paper gives a pseudo-workflow for deployment. In data preparation, one crawls or ingests articles, extracts hyperlinks per article, aggregates links per domain, builds the directed weighted graph, curates seed rewards, and tunes $s$5 or $s$6 by cross-validation with a preference for local homophily (Burdisso et al., 2024). In inference, one computes $s$7 for all sources, caches results, returns $s$8, label, and confidence for service calls, and provides ranked source lists or bulk-update endpoints when the graph or reward set changes (Burdisso et al., 2024).

Several downstream interfaces are explicit. Fake-news detection can use $s$9 as a prior or feature; fact-checking can prioritize claims from low-$\rho(s) < \rho(s')$0 sources; ranking and recommendation can discount items from low-$\rho(s) < \rho(s')$1 origins (Burdisso et al., 2024). The branch is also described as language-independent, with graphs constructed per language using fastText, and for unseen sources not yet present in the graph it returns $\rho(s) < \rho(s')$2 as unknown until sufficient interactions accumulate (Burdisso et al., 2024).

This suggests that the branch is not merely a scoring model but a reusable infrastructure layer centered on graph construction, propagation, and caching. That interpretation follows from the paper’s explicit separation between graph ingestion, reliability propagation, and downstream interfaces (Burdisso et al., 2024).

6. Limitations, robustness, and prospective extensions

The branch inherits several structural limitations. Sources must appear in the graph to be scored, so newly emerged domains may have sparse links; assigning $\rho(s) < \rho(s')$3 denotes unknown reliability (Burdisso et al., 2024). The reported experiments use an English graph because of ground-truth availability, even though the method itself is multilingual (Burdisso et al., 2024). CC-News also imposes temporal scope constraints, since domains that are only represented before 2016 may be absent (Burdisso et al., 2024).

The paper also notes that journalist labels emphasize factuality and may not capture political bias or press freedom considerations; reliability degrees should therefore not be treated as de facto values (Burdisso et al., 2024). Interaction data can be manipulated through coordinated link farming, and although the current approach relies on hyperlink proportions rather than social signals, it still requires safeguards such as filtering low-quality links and domain-level whitelists or blacklists (Burdisso et al., 2024). Domain shift remains a concern, and rerunning the pipeline on temporal snapshots is proposed as a way to maintain currency (Burdisso et al., 2024).

Transparency is presented as a strength of the approach: the graph and update equations are explicit, and the system is language- and content-independent (Burdisso et al., 2024). At the same time, the paper states that seed labels and graph construction choices can introduce bias and should therefore be documented (Burdisso et al., 2024).

Future work is described in several directions. Richer interaction modeling could add citation types, quote attribution, co-mention, and cross-platform signals such as Twitter, Facebook, and YouTube when accessible (Burdisso et al., 2024). Semi- or self-supervised extensions could infer seeds from weak signals or combine contrastive graph learning with RL propagation (Burdisso et al., 2024). Dynamic or temporal graphs could model reliability over time with discounting or aging (Burdisso et al., 2024). The paper also raises causal considerations, such as distinguishing endorsement from mere mentions and penalizing paid link schemes (Burdisso et al., 2024).

GNN integration is mentioned as a future possibility rather than a current component. The text explicitly states that GNNs are not used in the present paper, but future branches could combine RL-derived degrees with message passing and regularizers such as

$\rho(s) < \rho(s')$4

with careful treatment of the asymmetric influences revealed by FP-Reliability (Burdisso et al., 2024).

In its reported form, the branch scales to tens of thousands of sources and nearly a million edges, delivers continuous reliability degrees that correlate strongly with journalist scores, and provides a transparent, thresholdable module for real-world verification pipelines (Burdisso et al., 2024).