Threats to the sustainability of Community Notes on X (2510.00650v1)

Abstract: Community Notes are emerging as an important option for content moderation. The Community Notes system pioneered by Twitter, now known as X, uses a bridging algorithm to identify user-generated context with upvotes across political divides, supposedly spinning consensual gold from partisan straw. It is important to understand the nature of the community behind Community Notes, especially as the feature has now been imitated by several billion-user platforms. We look for signs of stability and disruption in the X Community Notes community and interrogate the motivations other than partisan animus (Allen, Martel, and Rand 2022) which may be driving users to contribute. We conduct a novel analysis of the impact of having a note published, which requires being considered "helpful" by the bridging algorithm, utilising a regression discontinuity design. This allows stronger causal inference than conventional methods used with observational data. Our analysis shows the positive effect on future note authoring of having a note published. This highlights the risk of the current system, where the proportion of notes considered "helpful" (and therefore shown to users on X) is low, 10%, and declining. This analysis has implications for the future of Community Notes on X and the extension of this approach to other platforms.

• The paper’s main contribution is establishing a 5% increase in continued contributions from first-time authors via regression discontinuity analysis.
• The study finds that a small fraction of highly active authors produce most helpful notes, while overall contributor retention steadily declines.
• Algorithmic filtering with strict diversity and rating thresholds reduces note publication, posing practical challenges for sustaining community engagement.

Sustainability Challenges in the Community Notes System on X

Introduction

The paper "Threats to the sustainability of Community Notes on X" (2510.00650) presents a comprehensive empirical analysis of the Community Notes system, a crowd-sourced content moderation mechanism originally launched as Birdwatch on Twitter (now X). The paper interrogates the stability, incentive structures, and long-term viability of the contributor community, leveraging open data and the bridging algorithm that underpins note publication. The authors employ regression discontinuity design (RDD) to establish causal relationships between note publication and contributor retention, providing robust evidence for the incentive effects of the system's publication threshold.

Community Growth and Contributor Dynamics

The expansion of Community Notes has been marked by rapid onboarding and subsequent stabilization of monthly active authors (MAAs), particularly following the global rollout in 2024. The system reached approximately 40,000 MAAs, with a much larger base of users engaged in rating notes.

Figure 1: Monthly Active Authors (MAAs) contributed at least one note during the reference month.

A pronounced skew in contribution is observed: a small fraction of authors are responsible for a disproportionate share of helpful notes. In 2024, the top 1% of authors produced 32% of all helpful notes, and the top 7.5% accounted for 50%.

Figure 2: Share of helpful notes authored by percentile of authors in 2024. Dashed line shows median (50th percentile) point.

This concentration of productive activity among a minority of contributors is consistent with other crowd-sourced moderation systems, but it raises concerns about resilience and vulnerability to churn.

Persistence and Churn in the Contributor Base

Longitudinal analysis reveals significant churn among note authors. Less than half of authors remain active one year after their first contribution, and persistence rates have declined over time.

Figure 3: Persistence rate of authors based on the year of the first note written.

The fraction of new authors who continue contributing within four months of their first note has also decreased, with only transient improvements during periods of program expansion.

Figure 4: Fraction of authors who stayed active (defined as authoring another note within 4 months of their first note).

These trends suggest that the onboarding pipeline is weakening, and the system's ability to maintain a healthy contributor base is increasingly dependent on retention rather than recruitment.

Algorithmic Filtering and Visibility of Notes

The bridging algorithm, which requires a diverse set of raters for a note to be deemed "helpful," results in most notes remaining unseen by the broader user base. The helpfulness score threshold (0.4) creates a sharp discontinuity: notes above this threshold are published, while those below are not, regardless of the number of ratings.

Figure 5: Note all-time highest helpfulness score according to the standard algorithm (“Max core note intercept”) against log Number of user ratings, colored by current note status, for 2024 notes.

The proportion of notes classified as helpful and published has declined since May 2024, exacerbated by a policy change in March 2025 requiring at least 10 ratings from users with different viewpoints.

Figure 6: Percentage of helpful notes. From March 2025, notes needed at least 10 ratings from users with different points of view to be considered helpful.

This reduction in publication rate has direct implications for contributor motivation and system sustainability.

Causal Analysis of Incentive Effects

The authors employ RDD to estimate the causal effect of note publication on subsequent author activity. By comparing first-time authors whose notes narrowly exceed or fall short of the publication threshold, they find that publication increases the probability of further contributions by approximately 5% (local average treatment effect = 0.052, $SE$ = 0.03, $p$ = 0.04).

Figure 7: Discontinuity in the relation between Final Note Score (Helpfulness) and author progression. Author progress is defined as 0 or 1.

Sensitivity analyses confirm the robustness of this effect to alternative window sizes and cutoff values. Notably, higher helpfulness scores above the threshold do not further increase retention rates, indicating a binary incentive structure. The analysis also demonstrates that while publication is a significant motivator, it is not the sole determinant of continued participation.

Implications and Future Directions

The findings highlight several risks to the sustainability of Community Notes:

• Declining Publication Rate: As the fraction of published notes decreases, the incentive for contributors to remain active is eroded, particularly for those whose notes consistently fail to meet the diversity threshold.
• Contributor Churn: The system is increasingly reliant on a small core of highly active authors, with declining rates of new author retention.
• Algorithmic Constraints: The bridging algorithm's emphasis on diversity, while effective in mitigating partisan bias, may inadvertently suppress the visibility of high-quality notes that lack sufficient cross-perspective ratings.
• AI Integration: The recent introduction of AI-authored notes raises questions about the future role of human contributors and the potential for competition or complementarity between human and machine-generated content.

From a theoretical perspective, the paper advances the application of causal inference methods in the analysis of crowd-sourced moderation systems. Practically, it underscores the need for platforms to balance algorithmic rigor with contributor incentives to ensure long-term viability.

Conclusion

The Community Notes system on X demonstrates both the promise and fragility of crowd-sourced content moderation. While the bridging algorithm effectively curates diverse, high-quality notes, the declining rate of note publication and contributor retention pose significant threats to sustainability. The causal link between publication and author persistence suggests that incentive structures must be carefully managed, especially as the system matures and expands to other platforms. Future research should explore alternative mechanisms for rewarding contributors, the impact of AI integration, and strategies for maintaining diversity and engagement in large-scale moderation communities.