The Effects of Request Alerts on the Diversity and Visibility of Community Notes

Abstract: Several major social media platforms have shifted toward crowdsourced fact-checking systems like Community Notes to combat misinformation at scale. However, these systems face criticism regarding which content is scrutinized and how visible that scrutiny is. To address these concerns, X allows users to request community notes for specific posts. When sufficient requests accumulate, X displays an alert, formalizing an interface cue intended to guide contributor behavior. In this study, we examine the effectiveness of request alerts. We infer the presence of request alerts at the time each note was written and identify 318 top writers who were repeatedly exposed to these alerts. Through analyzing their contributed 54,874 English notes written with and without request alerts, we find that at the individual level, writers fact-check more diverse and more political content under alerts. Nonetheless, at the collective level, these shifts direct contributions toward the already dominant Politics and Conflict category, thereby increasing content inequality within the Community Notes ecosystem. Finally, using a mixed-effects model that controls for both writer- and topic-level random effects, we estimate that notes written under alerts are between 8.4 and 20.2 percentage points more likely to be classified as helpful and thus visible to the public, compared to non-alerted notes. This visibility gain diminishes as topics diverge further from writers' prior interests, demonstrating a pivot penalty effect. Taken together, our findings show that request alerts function as an effective interface cue that increases both topical diversity and note visibility in Community Notes.

• The paper demonstrates that request alerts trigger individual topical exploration but concentrate attention on dominant categories system-wide.
• It employs advanced topic modeling with UMAP and HDBSCAN alongside causal inference to quantify topic shifts using cosine distances.
• Request alerts significantly boost note visibility by up to 20.2 percentage points, though this benefit diminishes as contributors diverge from their expertise.

The Effects of Request Alerts on Topical Diversity and Visibility in Community Notes

Introduction

This paper systematically analyzes the impact of request alerts within X's Community Notes platform, focusing on their effects on both the diversity of fact-checked content and the downstream visibility of contributed notes. It addresses serious concerns in crowdsourced fact-checking systems, including distributional biases in content scrutiny and the challenge of ensuring that high-quality fact-checks are surfaced to the public. The work leverages the large-scale public Community Notes dataset, develops topic modeling and causal inference pipelines, and provides empirical estimates of how interface cues—here, request alerts—restructure attention, contribution patterns, and promotional outcomes.

Data and Methodology

The study employs a longitudinal dataset that captures both writer activity and request dynamics. It reconstructs request alert exposure by algorithmically inferring when top writers would have seen request alerts, using writer activity history, request accrual times, and note submission logs.

Figure 1: Illustration of note publication status timelines, top writer status transitions, request alert inference methodology, and the mathematical quantification of topic shift $\Phi$.

For topical analysis, the authors aggregate note texts at the X-post level and apply Twitter4SSE model-based embeddings, dimensionally reduce with UMAP, and cluster with HDBSCAN, resulting in 239 fine-grained topics subsequently coded into seven broad categories.

Figure 2: Visualization of topic modeling and axial coding for 10,000 posts; intra-category coherence in embedding space.

Writers' "topic shift" at note creation is measured as the cosine distance between a note's embedding and the centroid of the writer's five most recent notes, quantifying the extent of pivoting from recent interests.

Effects on Fact-Checking Diversity

The analysis identifies clear micro- and macro-level effects regarding the diversity of topics addressed by contributors under request alerts.

Figure 3: (a) Bimodal vs. unimodal topic shift distributions for non-alerted versus request-alerted contributions; (b) topic shift increases as writers pivot away from their usual category (shown for Policy Violation).

Writers naturally exhibit a bimodal topic shift, balancing repeated topical engagement and occasional pivots. Under request alerts, this behavior is overruled: the zero-centered mode disappears, and contributors are systematically nudged to fact-check content outside their immediate past interests. However, this individual-level increase in exploration does not translate to higher system-wide diversity.

Figure 4: Propensity of writers to fact-check posts in each broad category with and without request alerts; notable increase for political content.

Request alerts significantly increase the individual likelihood of fact-checking political and controversial content and decrease engagement with "softer" topics, including Policy Violation and Science/Technology. This causes an aggregate shift: the politics/conflict category constitutes nearly half of request-alerted notes, further concentrating attention in already dominant topical domains.

Effects on Note Visibility

A central finding is the robust effect of request alerts on the probability that notes achieve "helpful" (CRH) status, the necessary criterion for public surfacing.

Figure 5: Marginal effect of request alerts on estimated ever-CRH rates by content category from a mixed-effects logistic regression; alerts boost visibility across all categories.

Request-alert exposure increases the chance of a note being ever rated helpful by 8.4 to 20.2 percentage points, depending on category, even after controlling for writer and topic random effects. The effect is strongest in Science/Technology but present throughout all domains.

Analysis of topic shift reveals a "pivot penalty."

Figure 6: Across all topic shifts, request-alerted notes are more likely to be CRH, but the gain erodes as contributors venture further from their recent topical center.

Visibility gains diminish as writers submit notes in less familiar topical regions: the probability increment conferred by request alerts decays as topic shift increases.

Theoretical and Practical Implications

The results highlight a key tension between micro-level and macro-level diversity. On an individual basis, request alerts act as attention cues and social signals, increasing the diversity of topics a writer engages with, consistent with interface cue theory and heuristic behavioral models [sundar2008main]. However, because community demand is unevenly distributed, system-level diversity decreases, concentrating effort in politics/conflict and exacerbating topical inequality.

Practically, this means that demand-driven signaling mechanisms like request alerts successfully direct contributors toward collectively salient or ambiguous posts, but may amplify existing saliency biases, increasing the risk that other misinformation genres remain under-scrutinized. The authors argue for balancing demand cues with supply-side or expert-driven routing—potentially algorithmic task assignment or targeted prompts for underrepresented issues.

The diminishing efficacy of request alerts with increasing topic shift exposes the limitations of attention redirection when decoupled from contributor expertise, a phenomenon analogous to the "pivot penalty" seen in interdisciplinary scientific work [hill2025pivot]. Task allocation mechanisms should consider not only where community attention is demanded, but also how to maximize effectiveness by matching alerts to contributors' topical competence regions.

Figure 7: Request alerts elevate currently CRH probabilities across all categories, consistent with ever CRH results.

Figure 8: The improvement of currently CRH rate also diminishes as writers move further from their previous interests.

Limitations and Future Directions

The paper notes important limitations. Topic modeling is based on concatenated note texts, not underlying post content, and cannot disambiguate between shifts in what is fact-checked and how it is described. The data lacks ground-truth about the factual complexity or verifiability of posts; thus, observed visibility gains may conflate increased contributor effort with the inherent fact-checkability of posts under alert.

Further research could incorporate richer platform-level signals, measure causal effects on misinformation coverage more directly, and develop hybrid task-routing mechanisms that integrate both demand-side alerts and supply-side (expertise-aware) recommender systems. The ultimate goal is optimizing both equity (diversity of addressed topics) and effectiveness (visibility of high-quality corrections) in participatory fact-checking infrastructures.

Conclusion

The paper provides robust evidence that request alerts in X's Community Notes systematically redirect contributor attention toward more politically salient and more diverse topics at the individual level, but induce further collective concentration in dominant categories. Request alerts unambiguously increase the probability that notes are surfaced as helpful, yet this effect decreases when contributors are pushed outside their recent focus—a measurable pivot penalty. These findings underscore the power and limitations of interface cues as collective coordination devices, with significant design implications for crowdsourced fact-checking systems that aim to balance responsiveness, expertise alignment, and equitable content coverage.