Community Notes on X: Crowdsourced Fact-Checking

• Community Notes on X is a crowd-sourced fact-checking system that enables users to append contextual corrections to misleading posts using community consensus and algorithmic assessment.
• It operates through a multi-phase workflow where user submissions are evaluated for helpfulness by diverse raters and selected based on cross-partisan agreement.
• Recent advances integrate large language models with human feedback to enhance note quality and speed, effectively reducing the spread of misinformation.

Community Notes on X is a crowd-sourced fact-checking system deployed on the social media platform X (formerly Twitter) as an intervention against the proliferation of misinformation. The mechanism enables platform users and, increasingly, automated systems to append context, provide corrections, and supply supporting evidence to potentially misleading posts. The system's design leverages human consensus, algorithmic diversity assessment, and, in recent developments, LLMs to scale the reach and speed of public information quality control. The following sections review the structural framework, operational principles, empirical impacts, algorithmic foundations, and recent advances in human-AI collaboration within Community Notes.

1. System Structure and Workflow

Community Notes operates as an open annotation layer on top of the X platform, where users propose contextual notes on posts they assess as misleading or lacking critical information (Franzmeyer et al., 5 Jun 2024). Any eligible user may write a note, typically including citations from external sources, factual clarifications, or correction of misleading claims. Proposed notes enter a multi-phase moderation pipeline:

• Submission Phase: Users submit notes, potentially referencing external, reputable sources.
• Community Evaluation: Notes are not immediately shown with the post. Instead, a pool of community raters—selected for diversity in historical ratings and demographic background—evaluate each note's helpfulness.
• Display Criteria: A note is attached to a post only if it accumulates a sufficient number of "helpful" votes from a diverse group. The selection algorithm reflects not just aggregate helpfulness but also cross-partisan agreement and avoidance of echo-chamber effects (Bouchaud et al., 18 Jun 2025). As of early 2024, fewer than 10% of submitted notes met the threshold for display (Franzmeyer et al., 5 Jun 2024).
• Continuous Feedback: Notes may be re-evaluated and removed or replaced if subsequent consensus shifts.

The effective operation of Community Notes depends on an algorithm that balances helpfulness consensus with diversity of rater perspective, aiming to privilege annotations recognized as useful across ideological divides.

2. Algorithmic and Statistical Foundations

The ranking and surfacing of Community Notes are governed by a matrix factorization model. Each rating by a user $r$ on note $n$ is modeled as:

$$\hat{\eta}_{rn} = \beta_0 + \beta_n + \beta_r + \theta_n \cdot \theta_r$$

where:

• $\beta_0$ is a global helpfulness baseline,
• $\beta_n$ is the intrinsic bias of note $n$,
• $\beta_r$ is rater $r$'s leniency,
• $\theta_n$ and $\theta_r$ are note and rater positions along a latent ideological spectrum (Bouchaud et al., 18 Jun 2025, Li et al., 30 Jun 2025).

Notes garnering "helpful" ratings from raters widely distributed along the ideological axis receive higher aggregate scores and are more likely to be displayed. This "bridging" algorithm withholds the most polarizing notes from display; only notes achieving cross-partisan consensus are surfaced, ensuring broad-based trust but leaving certain high-conflict posts unmoderated (Bouchaud et al., 18 Jun 2025).

A note’s "helpfulness" is not simply a function of votes, but of consensus diversity as formalized through the learned ideological dimension. Only about 12% of submitted notes reach public display (Bouchaud et al., 18 Jun 2025).

3. Empirical Impact and Limitations

Recent empirical studies provide causal evidence on the effectiveness and limitations of Community Notes (Chuai et al., 13 Sep 2024, Slaughter et al., 18 Feb 2025, Chuai et al., 2023):

• Reduction in Misinformation Spread: Displaying a Community Note reduces subsequent reposts of a misleading post by an average of 62.0%, with increased effects when notes appear earlier in the post’s lifecycle (Chuai et al., 13 Sep 2024). Analyses using synthetic control methods report post-note reductions of 45.7% in reposts, 43.5% in likes, 22.9% in replies, and 14.0% in views (Slaughter et al., 18 Feb 2025).
• Behavioral Change: Displayed notes can induce deletion; posts receiving a visible note are 103.4% more likely to be deleted by their authors (Chuai et al., 13 Sep 2024).
• Timing Constraints: The impact is strongly modulated by intervention speed. Only ~13.5% of displayed Community Notes appear before the swift "viral" phase of diffusion; real-world population-level reductions in misinformation spread are thus closer to 15% (Chuai et al., 13 Sep 2024). Alternative research finds no significant reduction in early engagement metrics (likes, retweets) where note posting lags the initial viral spread (Chuai et al., 2023).

Table: Engagement Reductions Attributed to Community Notes (Slaughter et al., 18 Feb 2025)

Metric	Post-Note Reduction (%)	Total-Lifespan Reduction (%)
Reposts	45.7	11.4
Likes	43.5	13.0
Replies	22.9	7.3
Views	14.0	5.7

The system effectively curtails the depth and structural virality of misinformation cascades but is limited by the delay in note surfacing and by design does not resolve the most polarizing content (Slaughter et al., 18 Feb 2025, Bouchaud et al., 18 Jun 2025).

4. Source Quality, Bias, and Public Agreement

Empirical source analysis shows most citations in Community Notes come from left-leaning news outlets and fact-checking organizations of high factuality (Kangur et al., 18 Jun 2024, Borenstein et al., 19 Feb 2025). Key findings include:

• Source Bias and Factuality: More than 50% of cited news sources are left-center in bias and rate highly on factuality. However, notes supporting the original post tend to cite right-leaning and lower-factuality sources more often (Kangur et al., 18 Jun 2024).
• Correlation with Perceived Helpfulness: Notes with high-factuality and more neutral or positive sentiment sources are rated as more helpful and garner higher public agreement (measured as the ratio of "Agree" to total ratings) (Kangur et al., 18 Jun 2024).
• Interdependence with Professional Fact-Checking: Effective community notes cite professional fact-checkers up to five times more frequently than previously reported. Notes tackling complex, broad, or conspiratorial misinformation narratives are twice as likely to include expert sources, underlining the intertwined role of professional and citizen fact-checking (Borenstein et al., 19 Feb 2025).

Agreement Index Formula (Kangur et al., 18 Jun 2024):

$$\text{Agreement} = \frac{\text{Number of "Agree"}}{\text{Number of "Agree"} + \text{Number of "Disagree"}}$$

5. Hybrid Human-AI Collaboration

Recent advances propose integrating LLMs into Community Notes in two major ways (Li et al., 30 Jun 2025, Mohammadi et al., 10 Jul 2025):

• Open Note Submission Ecosystem: Both human authors and LLMs may generate note drafts. All candidate notes—regardless of authorship—enter a unified pool where only human raters determine helpfulness, preserving community oversight. LLMs can identify potential misinformation, summarize evidence via web search, and quickly produce candidate notes at scale (Franzmeyer et al., 5 Jun 2024, Li et al., 30 Jun 2025).
• Reinforcement Learning from Community Feedback (RLCF): Human helpfulness ratings provide a reward signal for training LLMs to optimize for accuracy, neutrality, and broad appeal; LLMs are thus iteratively improved to match community standards (Li et al., 30 Jun 2025).

AI-generated feedback can further enhance note quality. Studies show that argumentative AI feedback (posing counterarguments) leads to the greatest improvements in subsequent human-authored notes, particularly in reducing partisan bias and encouraging balanced, contextualized contributions (Mohammadi et al., 10 Jul 2025).

Notable risks in the AI era include:

• Helpfulness Hacking: LLMs may optimize for surface-level popularity at the expense of factuality.
• Homogenization and User Demotivation: LLM-dominated note pools might crowd out human insight or generate unvaried output.
• Rater Overload: Large volumes of LLM-generated notes require scaling up rater capacity or prescreening pipelines (Li et al., 30 Jun 2025).

6. Global Deployment, Polarization, and Policy Implications

Community Notes now operates in multiple countries. Empirical scaling studies reveal that the system’s learned latent ideological dimension aligns with each country's principal axis of political polarization. However, by surfacing only those notes with cross-partisan consensus, the system is highly effective at moderating broadly agreed-upon misleading content yet systematically withholds notes where consensus cannot be reached—leaving the most polarizing claims unmoderated (Bouchaud et al., 18 Jun 2025).

Policy implications include:

• Scalability and Speed: Human-AI ecosystems allow rapid, wide coverage, but intervention must be accelerated, ideally preceding the viral phase of misinformation diffusion (Chuai et al., 13 Sep 2024).
• Balanced Fact-Checking Ecosystem: Platform moves to phase out professional partnerships may undermine effectiveness, as community notes are deeply intertwined with and dependent on established fact-checkers, particularly for complex or conspiracy-laden topics (Borenstein et al., 19 Feb 2025).
• Algorithmic Transparency and Bias Mitigation: Ongoing tuning of note selection (balancing cosensus and diversity), rater recruitment, and AI model alignment is necessary to minimize echo chambers and ensure equitable moderation (Kangur et al., 18 Jun 2024, Bouchaud et al., 18 Jun 2025).
• Ethical Considerations: Preserving human agency, ensuring transparency in AI interventions, and designing against automation bias are essential safeguards as automated systems take on larger roles in content moderation (Mohammadi et al., 10 Jul 2025).

7. Future Directions and Open Challenges

Critical research avenues include:

• Optimizing RLCF and LLM customization: Iteratively improving LLMs using collective human feedback while avoiding homogenization and bias (Li et al., 30 Jun 2025).
• Scalable Rater Infrastructure: Building open APIs and robust verification systems to support increased volume and diversity in human ratings.
• Combining Fact-Checking Modalities: Integrating community notes and professional fact-checking for complementary strengths, particularly on high-stakes misinformation (Borenstein et al., 19 Feb 2025).
• Dynamic Note Matching: Developing AI tools for adaptive reuse of validated notes across similar misinformation instances to maximize rater efficiency and minimize repetitive annotation (Li et al., 30 Jun 2025).
• Intervention Timing and Cascade Disruption: Prioritizing technical and sociotechnical solutions to deliver fact-checking interventions prior to or during the early viral stages (Chuai et al., 13 Sep 2024, Slaughter et al., 18 Feb 2025).

Ongoing transparent benchmarking (e.g., via HelloFresh (Franzmeyer et al., 5 Jun 2024)) and continuous updates to both human and machine moderation strategies remain central for maintaining the efficacy and public trust in Community Notes on X.