Papers
Topics
Authors
Recent
Search
2000 character limit reached

"That's AI Slop, You Bot!" Studying Accusations, Evidence, and Credibility in Online Discourse Towards LLM-Generated Comments

Published 10 Jun 2026 in cs.SI and cs.AI | (2606.12073v1)

Abstract: Generative AI has made fluent prose cheap to produce, breaking the old promise to readers that good writing meant real thinking. How have readers responded, and what can this tell us about changing anti-AI attitudes? We analyzed 25 million comments from Hacker News and Reddit (2023-2026), combining LLM judgment on 7,500 sampled accusations of AI use, sentiment trajectories, speech-act coding of 300 confirmed accusations of AI use, and a matched-control test of accused versus non-accused parent comments. We found that the pejorative-label share of accusations rose more than tenfold on both platforms while a placebo vocabulary of pre-2022 inauthenticity terms (shill, astroturf) did not. This shift reflected a fast-growing trend of branding any suspicious or seemingly inauthentic prose as "AI slop". The slop frame now constitutes 94 percent of pejorative mentions, with the dominant comments shifting in tone from mockery toward gatekeeping and structural protest. The key surprise comes from a matched-control test which found that prose features that statistically distinguish AI from human text do not predict which human text gets accused as AI. The new accusations work as social gatekeeping of perceived authenticity without actually screening for AI. This research extends signaling theory by showing that substitute signals used socially can grow even when inaccurate if the underlying detection problem cannot be solved at the non-expert level. It shows that AI's effects on writing from the reader side are distinct from those on the production (writer) side. Detection technology cannot resolve this dynamic because the social function of accusations is increasingly to perform social gatekeeping and in-group signaling as opposed to identifying AI-generated writing.

Authors (2)

Summary

  • The paper reveals that coordinated online accusations, centered on terms like 'AI slop', function as social gatekeeping rather than as reliable indicators of LLM-generated content.
  • Using a multi-method analysis of 25 million comments from 2023–2026, the study shows a tenfold rise in pejorative accusation frames across platforms.
  • The findings emphasize that community-driven accusations, detached from factual detection, drive epistemic injustice and demand trust-building measures over technical fixes.

Accusations and Social Gatekeeping in Online Discourse on LLM-Generated Texts

Introduction

The paper "That's AI Slop, You Bot! Studying Accusations, Evidence, and Credibility in Online Discourse Towards LLM-Generated Comments" (2606.12073) presents a comprehensive empirical study of how large-scale online communities—specifically Hacker News and Reddit—have responded to the proliferation of LLM-generated content. While prior literature has emphasized production-side impacts of generative AI, this work focuses on the reception-side: the emergence, consolidation, and function of accusations that online comments are AI-generated. The central finding is that, in response to the collapse of prose quality as a reliable indicator of authentic human effort, readers have coordinated on substitute accusation registers that perform significant social, rather than epistemically accurate, work.

Methodology and Data

The authors assemble a multi-method dataset comprising 25 million public comments from Hacker News and Reddit, covering 2023–2026 and stratified across diverse community types. Accusation candidates were extracted using a poly-tiered regex lexicon spanning direct accusations, pejorative labels ("AI slop", "GPT garbage"), stylistic-tell callouts, parody/mocking, and indirect identifications. Rigorous validation was performed on 7,500 sampled accusations via per-comment LLM (Claude Opus 4.7) adjudication, and a matched-control design compared prose features of accused versus non-accused comments. Further, sentiment analysis and manual speech-act coding were used to quantify affective and pragmatic migration of accusation forms over time.

Key Findings

Lexical and Pragmatic Consolidation

A core result is the rapid and near-total lexical consolidation around the “AI slop” frame in pejorative accusations. On both Reddit and Hacker News, the share of pejorative accusations increased more than tenfold from early 2023 to 2026; the "slop" label alone captured 94% of all pejorative mentions by 2026, with older derogatory terms (“garbage”, “drivel”, etc.) displaced almost entirely. The accusation register thereby optimized for low-cost, high-recognizability coordination.

The share of such accusations grew stably and in parallel across both platforms, reaching approximately 24–26% of all candidate accusation comments by early 2026, with platform-specific inflections reflecting userbase composition and adoption timing. This trajectory was robust to placebo falsification: unlike pre-LLM inauthenticity vocabulary (“shill”, “astroturf”), which declined or remained flat, the “slop” register grew exponentially, confirming specificity to the LLM era.

Detection Versus Social Function

Crucially, despite the high coordination on accusation language, matched-control analysis revealed that human-accused comments possessed none of the statistical markers (article density, contraction rate, formal adverb use, mean token length, etc.) that robustly distinguish LLM-generated from human comments. Logistic regression found no significant predictive power of such markers for actual accusations, with the sole exception of marginally significant formal-register adverb use (p = 0.083).

This misalignment demonstrates that community accusations function primarily as social gatekeeping mechanisms, not as accurate detection tools for AI-generated content. Accusations serve to police boundaries of perceived authenticity, enact status signaling, and defend occupational or community jurisdictions, independent of the actual empirical presence of LLM-generated text.

Sociolinguistic Dynamics and Enregisterment

The accusation register’s consolidation was not dependent on formal governance. Both platforms, regardless of explicit anti-AI posting rules, saw parallel emergence and stabilization of accusatory norms, supporting a grassroots-prescriptivist rather than top-down enforcement model. The stabilization also occurred with high temporal compression; what in dialect or slang evolution takes decades, here unfolded over ~18–24 months, signifying a sociotechnical acceleration of enregisterment processes.

Speech-Act Migration and Affective Hardening

Speech-act analysis showed a shift from early mocking/dismissive accusations towards structural protest and explicit gatekeeping by 2026. The share of accusations categorized as structural protest (objections to generative AI as a phenomenon) tripled (to ~39%), while gatekeeping (claims of rule enforcement or community boundary policing) rose from 1.9% (2023) to 16.5% (2026). “Sneer” and parody forms remained stable or diminished, replaced by more institutionalized or political framing. Sentiment analysis further established a hardening affect: the mean compound sentiment of accusations dropped, and strongly negative accusations doubled across the study window.

Theoretical Implications

Signaling Theory Extension

The findings challenge and extend classic signaling theory. While theory predicts that, following a collapse in the costliness (and thus reliability) of a signal (e.g., good prose), populations will coordinate on harder-to-fake alternatives, empirical results here show that the substitute need not achieve actual screening accuracy to persist. Substitute signals can stabilize and perform substantial social work purely on the basis of their utility for in-group signaling and boundary maintenance.

Social Epistemology and Epistemic Injustice

This work empirically documents an inversion of epistemic injustice narratives. Instead of LLMs as perpetrators of testimonial injustice, coordinated lay-user accusations themselves produce systematic credibility deflation of authentic human writers, not because of writer identity but due to perceived (and unverified) authorship method. The harm emerges from structural incapacity (community-level inability to accurately detect LLM-authorship) rather than from explicit intent or institutional authority.

Sociology of Cultural Production

The accusation register functions as contemporary boundary work—a mechanism for communities of text labor (e.g., writers, programmers) to defend craft legitimacy under technological disruption. However, the actual signals used are decoupled from factual detection and become symbolic tokens for community policing. This boundary work is dynamically moving from informal sneering to formalized rule enforcement, making the performative aspect of anti-AI signaling a microcosm of wider professional realignments under generative AI pressure.

Practical and Policy Implications

For online platforms, explicit anti-AI content policies and moderation cannot address the underlying problem: accusation energy is redirected, not resolved, by rule changes, and accusation registers are set by user coordination rather than corporate fiat. High-quality engagement norms (as in r/changemyview) are the only observed structures that mitigate indiscriminate accusation drift.

For writers, the risk of being accused of AI generation is not a function of writing in any empirically LLM-like way but of prevailing community prejudice and miscalibrated heuristics. This imposes epistemic and reputational costs not tied to actual behavior. For policymakers, investment in epistemic infrastructure and trust mechanisms—not merely in improved detection technologies—is required. Since the accusation register’s social utility is decoupled from technical detection, technical solutions alone are insufficient.

Limitations and Future Directions

The study’s lexicon was tailored to English and major Anglophone platforms; cross-linguistic transfer and non-Reddit/Hacker News forums remain untested. Writer-side behavioral adaptation to accusation events requires further panel research. Broader generalization to image, voice, or code AI-authorship signaling is hypothesized but awaits empirical validation.

Conclusion

Between 2023–2026, Discord communities coordinated on a substitute accusation register (“AI slop”) in response to generative LLMs’ collapse of prose as a signal of human effort. This register consolidated rapidly and performed significant social functions—gatekeeping, status policing, and structural protest—without developing any real detection accuracy. Substitute signals in information-asymmetric environments may thus survive and propagate due to social rather than epistemic selection pressures, producing new forms of testimonial injustice and boundary-making. Future AI detection and governance models must directly reckon with the social, not just the technical, work of lay accusation and signaling in online communities.

Reference:

"That's AI Slop, You Bot! Studying Accusations, Evidence, and Credibility in Online Discourse Towards LLM-Generated Comments" (2606.12073)

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Explain it Like I'm 14

A simple explanation of the paper

What is this paper about?

This paper looks at how people online react to writing they think was made by AI (like ChatGPT). It studies how often people accuse others of using AI, the words they use (for example, calling it “AI slop”), and whether those accusations are actually good at spotting AI-written text. The authors analyzed a huge number of comments from two big sites, Reddit and Hacker News, between 2023 and 2026.

What questions were the researchers trying to answer?

The researchers asked a few main questions, in plain terms:

  • After AI tools became common, did readers switch to a new way of judging if something feels “truly human-made”? If so, what does that new signal look like?
  • Are these new signals (like calling something “AI slop”) any good at actually catching AI-written text?
  • How fast did this new way of talking appear and spread? Did it happen because of official rules, or because lots of users copied each other?
  • What’s the social purpose of these accusations? Are people trying to protect community quality, show status, protest AI in general, or something else?

How did they study it? (Methods in everyday language)

The team gathered about 25 million online comments from:

  • Hacker News (a tech discussion site)
  • 18 different Reddit communities (some about AI and tech, some about art and writing, some general discussion)

They looked for comments that sounded like AI accusations using a big list of patterns (like “this is AI,” “ChatGPT wrote this,” “AI slop,” or mocking phrases that imitate AI). Think of it like using a very detailed search to find all the ways people might accuse writing of being AI-generated.

Because some matches can be wrong, they took thousands of these “found” comments and had a strong AI system read them one by one to judge whether they were real accusations or something else (like just talking about AI, or a false alarm). This is like taking a careful sample to double-check the automatic search.

They also:

  • Measured the mood of accusation comments over time (more positive or negative).
  • Labeled what kind of speech each accusation was (quick sneer, dismissal, parody, gatekeeping about community rules, or broad complaints about AI).
  • Used a “placebo” check: they tracked old words for fakeness that existed before modern AI (like “shill” or “astroturf”). If all suspicion was rising, those old words would rise too. If only AI-accusation language climbed, that’s a clue something special changed.
  • Ran a “matched-control test” to see if accused comments actually look more “AI-like” than similar comments that weren’t accused. Imagine comparing a comment that got accused to a bunch of other comments from the same place and time with similar length, then checking if writing-style clues differ.

About those “writing-style clues”: they checked simple signals such as:

  • Contractions (don’t vs. do not)
  • Fancy adverbs (however, therefore)
  • Prepositions (of, in, to)
  • How varied the sentence lengths are
  • Average word length

These are everyday, measurable features that often differ between AI and human writing.

What did they find, and why does it matter?

Here are the key takeaways:

  • Accusations exploded and centered on one phrase: “AI slop.”
    • On both Reddit and Hacker News, accusations using pejorative labels (especially “slop”) shot up from a tiny share to about a quarter of all accusation-like comments by 2026.
    • Older “fake” words (like “shill”) did not rise. That means people weren’t just getting more suspicious generally; they were specifically using a new AI-focused language.
    • The word “slop” took over: by 2026, it made up the vast majority of insult-style AI accusations.
  • The tone and purpose of accusations changed.
    • Early on, people mocked or joked about AI writing.
    • Over time, more comments became about gatekeeping (“this doesn’t belong here”) and wider protests against AI in writing.
    • Moderator rule-enforcement messages started using the same language, showing that what began as user talk turned into community rules and actions.
  • Surprise: Accusations don’t actually track AI-like writing features.
    • The team showed that certain style clues do separate clearly AI-written text from human-written text (for example, AI tends to have more formal adverbs, fewer contractions, and more variable sentence lengths).
    • But when they compared accused human comments to similar non-accused human comments, those AI-like clues did not predict who got accused.
    • Translation: People are accusing based on vibes, context, and quick signals—not on the real telltale features that would reveal AI.
  • This is social gatekeeping, not accurate detection.
    • The phrase “AI slop” works like a badge or warning sign that communities use to police what they think is “authentic.”
    • It helps people coordinate quickly (short, catchy, recognizable), even if it’s often wrong.
    • Because the accusation serves a social function (boundary-keeping and group identity), better AI detectors won’t fix the problem by themselves. The accusations aren’t mainly about accuracy to begin with.
  • Platform differences exist but tell the same story.
    • Hacker News was a bit ahead of Reddit, probably because its tech-savvy users met AI tools earlier.
    • Most subreddits ended up showing the same pattern, except places with strong norms of thoughtful discussion (like r/changemyview), where accusations stayed relatively low.

What does this mean in the bigger picture? (Implications)

  • For readers and communities:
    • Be careful: accusing someone of using AI is often a social reaction, not a reliable test.
    • Strong community norms (like encouraging thoughtful dialogue) may reduce knee-jerk accusations.
  • For writers:
    • Writing that looks polished and formal can draw accusations even if it’s 100% human. Accusations don’t reliably match AI-like style.
  • For platforms and moderators:
    • Rules that encourage good-faith discussion can channel or reduce accusation “drift.”
    • Detection tools alone won’t fix social gatekeeping. It helps to invest in norms and clear, fair processes.
  • For theory and future research:
    • In classic “signaling theory,” when a trusted signal breaks (like “good writing = real effort”), people should find a new, harder-to-fake sign. Here, users did find a new signal (“AI slop”), but it isn’t accurate.
    • This shows that when regular people can’t really solve the detection problem, a substitute signal can still spread and stick because it serves social goals (status, boundaries, protest).
    • Expect similar patterns in other media (images, voice, code): public accusations may grow for social reasons even if they’re poor at spotting AI.

In short

The internet has developed a fast, catchy way to call out suspected AI writing—“AI slop.” That label spread quickly and now dominates how people talk about suspicious text. But these accusations don’t actually identify AI reliably. Instead, they mostly act as social tools to guard community boundaries and express frustration with AI. Because the accusations are about group norms more than truth, better AI detectors won’t solve the problem on their own. Building healthier discussion norms matters more.

View Paper Prompt  View All Prompts 

Knowledge Gaps

Unresolved knowledge gaps, limitations, and open questions

Below is a single, consolidated list of concrete gaps and open questions that future research can address.

  • Ground-truth authorship is absent: establish datasets with verified human- and LLM-generated comments (e.g., platform-verified disclosures, controlled posting experiments) to quantify true/false positive rates of lay accusations.
  • Indirect screening-accuracy test: expand beyond six shallow proxies by evaluating richer stylometric and discourse features (e.g., syntactic complexity, burstiness, hedging, discourse markers, perplexity across multiple LMs, stylometric embeddings, watermarking signals) against accusation likelihood.
  • Matched-control design scope: replicate the RQ2 test including top-level posts and cross-thread controls, and incorporate propensity-score matching on topical and temporal covariates to reduce selection bias.
  • Missing user-level covariates: analyze whether account age, karma, participation history, flair, or posting frequency predict both making and receiving accusations; identify “serial accusers” and “high-risk” targets.
  • Demographic and identity harms: test whether accusations disproportionately target non-native English writers, newcomers, or users inferred (cautiously and ethically) to belong to marginalized identities.
  • Moderator vs user dynamics: quantify the share and evolution of gatekeeping attributable to moderators (templated removals) versus ordinary users; test whether mod interventions crowd-in or crowd-out user accusations.
  • Causal effects of governance changes: run difference-in-differences or event-study designs around the introduction of explicit AI rules at the subreddit level to estimate causal impacts on accusation rates and tones.
  • Downstream consequences: measure how accusations affect thread outcomes (scores, deletions, lock rates), author behavior (editing, disclosure, exit/retention), and future accusations in the same threads.
  • Topic heterogeneity: apply topic modeling to identify domains (e.g., politics, health, programming help) with elevated accusation rates and test whether social functions differ by topic.
  • Cross-platform generalizability: extend the analysis to other venues (X/Twitter, Facebook groups, Stack Overflow, Discord, YouTube, blogs/comments) to see if the “slop” register diffuses similarly.
  • Cross-lingual scope: map and study analogous pejorative frames in non-English communities; build multilingual lexicons and evaluate whether enregisterment/compression occurs at similar tempos.
  • Temporal durability: continue tracking post–May 2026 to assess whether the “slop” frame persists, fragments, or is replaced as LLM outputs evolve.
  • Placebo coverage: broaden “inauthenticity” placebos (e.g., “NPC,” “copypasta,” “brigade,” “spamfarm,” “LLMbro”) to ensure declines in pre-2022 terms don’t mask growth in other non-AI suspiciousness frames.
  • Inflection-point causality: link the three trajectory inflections to exogenous events (press coverage spikes, major model releases, platform rule changes) via formal event studies.
  • Enregisterment mechanics: identify seed communities and pathways of lexical diffusion for “slop” using network contagion models; test whether a small number of high-centrality users catalyzed adoption.
  • Reliability of LLM judgment: report and improve validation of LLM-based labeling by adding multi-annotator human adjudication with inter-rater reliability (e.g., Cohen’s/Fleiss’ kappa) and cross-model agreement tests.
  • Sentiment measurement: replace or augment VADER with sarcasm- and irony-aware models fine-tuned on Reddit/HN data; validate sentiment against human ratings to ensure “sneer” and parody tones are captured.
  • Speech-act coding transparency: provide codebooks, annotator training materials, and reliability statistics; test whether speech-act distributions replicate with independent human coders and across platforms.
  • Coverage/precision of regex lexicon: audit misses (false negatives) and context failures (e.g., “slop” not referring to AI) with systematic error analyses; complement with supervised classifiers to improve recall/precision.
  • Data completeness: account for the impact of deleted/edited/removed comments (often missing from archives); integrate moderation logs where available or collect streaming data to reduce survivorship bias.
  • Bot and automation roles: detect whether bot accounts are significant accusers or targets; quantify their contribution to accusation dynamics and linguistic uniformity of the “slop” register.
  • Actual AI prevalence: independently estimate the share of AI-generated comments over time (e.g., via controlled honeypots or platform-side disclosures) to contextualize accusation volume relative to true AI use.
  • Heterogeneity in effects: examine whether the matched-control null holds uniformly across subs, topics, and time, or if there are niches where accusations better track AI-like features.
  • Unexplained predictors: investigate why mean token length negatively predicts accusation (OR=0.78) while other proxies do not; test nonlinearities and interactions (e.g., length × topic).
  • Writer-side strategies: experimentally test whether adding idiolectal quirks, personal anecdotes, or disclosures reduces accusation risk—and whether such strategies inadvertently become new “tells.”
  • Reader motivation: complement observational data with surveys/interviews/experiments to unpack perceived goals (quality control, status marking, political protest) and willingness to update when corrected.
  • Intervention design: run field experiments (e.g., prompts nudging substantive engagement, friction for pejorative replies, verified-disclosure badges) to measure effects on accusation rates and community health.
  • Legal/policy implications: assess the risks of defamatory or harassing accusations at scale and the efficacy of policy tools (appeals, transparency reports, right of reply) in mitigating harm.
  • Modality transfer: empirically test whether analogous accusation registers emerge for images, audio, and code; compare dynamics and accuracy across modalities where detection baselines differ.
  • Replicability and openness: ensure code, lexicons, and labeled subsets are released (with privacy safeguards) to enable independent replication and robustness checks across alternative archives and sampling schemes.
  • LLM labeling drift: evaluate whether the classifier model’s judgments are stable over time and domains (model updates may change outputs), and calibrate with periodic human re-annotation.
View Paper Prompt  View All Prompts 

Practical Applications

Overview

Below is a set of practical, real-world applications grounded in the paper’s findings that AI-use accusations in online discourse increasingly function as social gatekeeping rather than accurate detection. Applications are grouped by deployment timeline and include sector links, candidate tools/workflows, and feasibility assumptions.

Immediate Applications

  • Community “accusation register” monitoring and alerts
    • Sectors: Software/platforms; Online communities; News/media; Open-source communities
    • Action: Track the Tier-2 pejorative (“slop”) share, placebo inauthenticity lexicon, sentiment, and speech-act mix as leading indicators of gatekeeping drift and community health.
    • Tools/workflows:
    • Moderation dashboards that chart T2 share, VADER sentiment, and per-tier “REAL” rates by forum/sub-month.
    • Alerts when pejorative-share or negative sentiment breaches thresholds.
    • Assumptions/dependencies: Access to comment streams/metadata; acceptance of VADER/LLM-assisted labeling; metric calibration by community.
  • Moderation policy updates that decouple “AI-use” accusations from removal decisions
    • Sectors: Software/platforms; News/media; Education forums; Developer forums
    • Action: Rewrite rules to require content-based critique or specific rule violations (e.g., plagiarism, misinformation) rather than authorship-method claims; classify “AI slop” without evidence as low-value/harassing speech.
    • Tools/workflows:
    • Policy templates: “Evidence or substance, not authorship” guidelines.
    • Moderator checklists requiring a content-quality rationale before removal.
    • Assumptions/dependencies: Community buy-in; legal/ToS alignment; moderator training.
  • Friction and structured-reply UX for pejorative-labeled accusations
    • Sectors: Social platforms; Comment systems; Knowledge forums
    • Action: When users type terms from the pejorative lexicon (“slop,” “GPT garbage”), inject an inline nudge to:
    • Provide specific evidence or switch to report flow.
    • Use a structured critique form (e.g., “Which claim is inaccurate? Provide a source.”).
    • Tools/workflows:
    • Client-side input filters with just-in-time nudges.
    • Template-based “constructive critique” forms.
    • Assumptions/dependencies: False-positive tolerance for language filters; A/B testing capacity.
  • Moderation triage that redirects “accusation energy”
    • Sectors: Platforms; Enterprise community support; Newsrooms
    • Action: Route AI-use accusations to a dedicated queue with guidance to evaluate policy-relevant harms (spam, harassment, misinformation) rather than “AI/not-AI.”
    • Tools/workflows:
    • Triage labels (“accusation,” “quality,” “policy breach”).
    • Auto-replies pointing to disclosure rules and content standards.
    • Assumptions/dependencies: Workflow integration with existing mod tooling; staffing.
  • Community norm strengthening modeled on r/changemyview
    • Sectors: Platforms; Knowledge forums; Professional associations
    • Action: Adopt upfront norms that incentivize substance (e.g., citation requirements, steelmanning rules, AI-use disclosure norms) to reduce the payoff to gatekeeping.
    • Tools/workflows:
    • Template rulesets; “Top-of-sub” stickies explaining evidence-based norms.
    • Reward systems (badges/karma) for substantive engagement.
    • Assumptions/dependencies: Willingness to enforce norms; clear appeals process.
  • Brand, PR, and customer-support playbooks for AI-use accusations
    • Sectors: Enterprise; Consumer platforms; SaaS
    • Action: Train teams to defuse accusations by:
    • Addressing the content, not alleged authorship.
    • Disclosing AI assistance when used, with human-in-the-loop explanations.
    • Offering revision or sources rather than debating “AI/not-AI.”
    • Tools/workflows:
    • Response templates; escalation SOPs for persistent gatekeeping.
    • Tagging incidents to monitor accusation volume over time.
    • Assumptions/dependencies: Clear internal AI-use policies; legal/compliance review.
  • Academic integrity and pedagogy adjustments that focus on process evidence
    • Sectors: Education; EdTech
    • Action: Prioritize assignment designs and grading rubrics that reward process artifacts (drafts, version histories, oral defenses) instead of relying on surface prose as a quality signal.
    • Tools/workflows:
    • Portfolio-based assessment; version-control submissions; viva-style checkpoints.
    • Transparent AI-use disclosure forms outlining allowed/forbidden assistance.
    • Assumptions/dependencies: Instructor workload; LMS support; student buy-in.
  • Legal/compliance guidelines for moderation risk
    • Sectors: Platforms; Media; Enterprises hosting user content
    • Action: Update risk frameworks recognizing that AI-use accusations often misfire; reduce defamation and wrongful removal exposure by tying enforcement to content harms.
    • Tools/workflows:
    • Legal review of policies; standardized moderator rationales logged per action.
    • Assumptions/dependencies: Jurisdictional variance; safe-harbor considerations.
  • Research replication kits for discourse analysis
    • Sectors: Academia; Think tanks; Platform research teams
    • Action: Reuse the paper’s lexicon-tier approach, LLM validation, and matched-control design to monitor other communities or languages.
    • Tools/workflows:
    • Open-source regex lexicons; labeling protocols; analysis notebooks.
    • Assumptions/dependencies: Data access; IRB/ethics approvals; cross-lingual adaptation.

Long-Term Applications

  • Cross-modal gatekeeping monitors (text, images, audio, code)
    • Sectors: Platforms; Developer ecosystems; Creative marketplaces
    • Action: Extend the “accusation register” approach to image authenticity, voice-cloning, and code authorship, where social gatekeeping is likely to emerge similarly.
    • Tools/workflows:
    • Multimodal lexicons; speech-act classifiers; trend dashboards per medium.
    • Assumptions/dependencies: Availability of modality-specific training data; evolving norms per medium.
  • Evidence-based critique protocols and community governance redesign
    • Sectors: Platforms; Professional societies; Open-source foundations
    • Action: Institutionalize “evidence-first” adjudication (content harm > authorship suspicions) with audit trails and community juries or ombuds functions.
    • Tools/workflows:
    • Case-management for contested posts; peer-review style evaluations for removals.
    • Assumptions/dependencies: Governance legitimacy; resource allocation; transparency norms.
  • Author provenance and attestation ecosystems (with careful incentives)
    • Sectors: Media; Academia; Enterprise content; Software
    • Action: Develop opt-in provenance/attestation standards (e.g., C2PA-like signatures, human-in-the-loop attestations) to settle disputes when necessary, while acknowledging they won’t neutralize gatekeeping motives.
    • Tools/workflows:
    • Signing tools embedded in CMS; verifier services; selective disclosure protocols.
    • Assumptions/dependencies: Interoperability, privacy concerns, partial adoption; attacker models.
  • Third-party verification and arbitration services for contested authorship
    • Sectors: Media; Legal-tech; Marketplaces; Hiring platforms
    • Action: Offer neutral assessments when accusations threaten reputations or contracts, combining process evidence, version histories, and expert review rather than unreliable detectors.
    • Tools/workflows:
    • Chain-of-custody capture (draft logs), expert panels, standardized reports.
    • Assumptions/dependencies: Cost and turnaround; acceptance by platforms or courts.
  • Large-scale public literacy campaigns on AI detection limits
    • Sectors: Public policy; Education; Media literacy NGOs
    • Action: Reduce overreliance on lay “tells” by teaching why detection is hard and how to critique substance instead.
    • Tools/workflows:
    • Curriculum modules; platform PSAs; educator toolkits.
    • Assumptions/dependencies: Funding; sustained engagement; measurement of impact.
  • Adaptive moderation models that anticipate accusation waves
    • Sectors: Platforms; Online communities
    • Action: Predict spikes in pejorative accusation during model releases/news cycles and pre-position moderators, automated nudges, and sticky threads clarifying rules.
    • Tools/workflows:
    • Time-series forecasting tied to AI-news signals; surge playbooks.
    • Assumptions/dependencies: Reliable signal sources; elastic moderation capacity.
  • Hiring and evaluation frameworks that devalue surface-prose proxies
    • Sectors: HR/Recruiting; Academia; Professional certification
    • Action: Shift evaluation from polished prose to demonstrations, interviews, and work-sample tests with process verification to mitigate both AI-use and gatekeeping biases.
    • Tools/workflows:
    • Scenario-based assessments; monitored coding/writing sessions; artifact audits.
    • Assumptions/dependencies: Cost/time; candidate experience; fairness audits.
  • Platform-level “gatekeeping speech” classifiers integrated with harm policies
    • Sectors: Platforms; Trust & Safety vendors
    • Action: Train models to detect gatekeeping speech acts (e.g., pejorative labeling without evidence) and route to soft interventions or policy enforcement.
    • Tools/workflows:
    • Speech-act classifiers; configurable response ladders (nudge → limit → removal).
    • Assumptions/dependencies: Model accuracy across communities; appeal mechanisms; context sensitivity.
  • Comparative, cross-lingual studies and tools
    • Sectors: Academia; Global platforms
    • Action: Build multilingual lexicons and culturally tuned models to track how “AI slop”-like registers enregister and evolve across languages.
    • Tools/workflows:
    • Community translation pipelines; local researcher partnerships; benchmark datasets.
    • Assumptions/dependencies: Cultural variation; resource languages vs. low-resource contexts.
  • Policy frameworks that recognize social gatekeeping dynamics
    • Sectors: Regulators; Standards bodies; Digital governance
    • Action: Issue guidance discouraging reliance on automated “AI detectors” for enforcement; encourage content-focused standards and due-process in moderation.
    • Tools/workflows:
    • Model policy templates; audits of platform enforcement tied to measurable harms.
    • Assumptions/dependencies: Regulatory jurisdiction; industry consultation; civil society input.

Key assumptions and dependencies across applications

  • Social function over detection: Since accusations serve gatekeeping/status signaling, tools must target behavior and incentives, not just improve AI detection.
  • Evolving language: Lexicons will change (e.g., “slop” dominance today); monitoring and models need continuous updating.
  • Generalization limits: Findings are from English-language Reddit/Hacker News (2023–2026); cross-platform and cross-cultural validation is required.
  • Measurement choices: VADER and LLM judgments introduce their own biases; mixed-method audits and human oversight remain important.
  • Community legitimacy: Durable change depends on norm-setting, transparency, and fair appeals—not solely automated enforcement.
View Paper Prompt  View All Prompts 

Glossary

  • Affective hardening: A shift toward more negative or intensified emotional tone in a discourse over time. "Affective hardening was tested through three independent procedures."
  • Boundary maintenance: The preservation of social or community boundaries through practices that reinforce who belongs and who does not. "accusations like 'AI slop' spread because they serve social functions such as boundary maintenance, status signaling, and gatekeeping"
  • Boundary work: Sociological processes by which groups demarcate domains of expertise and authority. "Boundary work can help us understand these issues, whereby professions compete for jurisdictional control over domains of practice"
  • Chi-square: A statistical test that compares observed and expected frequencies to assess association. "A 2x2 chi-square comparing T2 versus placebo composition in 2023 against 2026 on Reddit yields chi-square = 11,116 on 1 degree of freedom"
  • Degraded equilibria: Market outcomes where low-quality producers dominate due to information problems. "markets with information asymmetry between buyers and sellers tend toward degraded equilibria where bad-quality producers dominate"
  • Enregisterment: The sociolinguistic process by which linguistic features become recognized as a distinct register linked to social meanings. "Cumulative stance-taking by community members aggregates into what is called enregisterment."
  • Enregisterment theory: A framework explaining how sets of linguistic features become conventionalized to index social identities or stances. "Enregisterment theory describes how a set of features is recognized as a coherent variety indexing particular speakers or stances"
  • Epistemic injustice: Harm done to someone in their capacity as a knower, such as through unfair credibility deficits. "Moreover, epistemic injustice is harm when a speaker is wronged in their capacity as a knower"
  • Gatekeeping: Regulating access to a community or discourse, enforcing norms about who or what is acceptable. "the social function of accusations is increasingly to perform social gatekeeping and in-group signaling"
  • Grassroots-prescriptivism: Bottom-up enforcement of language or community norms by ordinary users rather than formal authorities. "the finding supports a grassroots-prescriptivism account over a governance-driven one."
  • Hermeneutical injustice: A form of epistemic injustice arising from collective gaps in interpretive resources that disadvantage certain speakers. "This may sit closer to hermeneutical injustice: accusers lack the interpretive resources to accurately assess whether a text is AI-generated,"
  • Information asymmetry: Situations where one party in a transaction has more or better information than another. "markets with information asymmetry between buyers and sellers"
  • Logistic regression: A statistical model for predicting a binary outcome (e.g., REAL vs not) from explanatory variables. "A logistic regression of P(REAL) on tier, year, and sub-cluster on the 5,000-comment Reddit sample confirms the picture at the population level."
  • Mann-Kendall trend test: A nonparametric test used to assess the presence of a monotonic trend in a time series. "Mann-Kendall trend tests on the full 41-month series confirm the directionality formally."
  • Mann-Whitney U test: A nonparametric test comparing distributions of two independent samples. "Mann-Whitney U tests confirm that DISCLOSURE differs from REAL on formal register (p < 1e-9), preposition density (p < 1e-11), sentence variance (p < 1e-15), and mean token length (p < 1e-15)."
  • Matched-control test: A design that compares treated units to similar control units matched on key characteristics. "a matched-control test of accused versus non-accused parent comments."
  • Metalinguistic speech acts: Utterances about language itself that evaluate or regulate how language is used. "A complementary account describes how ordinary speakers participate in defining language through cumulative metalinguistic speech acts"
  • Odds ratio (OR): A measure of association indicating how the odds of an outcome change with a predictor. "Tier 2 is 5.4 times more likely to be a REAL accusation than Tier 1 holding year and cluster constant (OR = 5.40, z = 16.2, p < 1e-15)."
  • OLS regression: Ordinary Least Squares regression; a method estimating relationships between variables by minimizing squared errors. "An OLS regression of compound on tier and year shows that all four tier dummies are significantly more positive than the T1 reference"
  • Pejorative: Expressing contempt or disapproval; here, derogatory labels used in accusations. "The slop frame now constitutes 94 percent of pejorative mentions,"
  • Placebo design: An analysis strategy using a parallel but conceptually adjacent measure to test for spurious effects. "The placebo design rules out generalized suspicion,"
  • Placebo lexicon: A control set of terms not related to the primary phenomenon, used to test specificity. "A 14-pattern placebo lexicon of pre-2022 inauthenticity vocabulary (shill, astroturf, sockpuppet, paid shill, fake account, corporate shill, talking points, payola, and variants) was applied"
  • Pseudo-R2: A goodness-of-fit measure for models like logistic regression, analogous to R2 in linear regression. "Pseudo-R2 0.150"
  • Signaling theory: Economic theory describing how agents convey information credibly via costly signals. "signaling theory predicts is necessary for a counter-signal to survive?"
  • Speech act: An utterance that performs an action (e.g., accusing, asserting) in communication. "The speech act evaluates the writing, positions the speaker against the writer, and claims community membership and gatekeeping in one short move."
  • Stance theory: A framework analyzing how speakers position themselves, others, and objects through evaluative language. "Stance theory describes the individual speech-act mechanism"
  • Structural protest: A community-level objection targeting broad phenomena or systems rather than individual instances. "STRUCTURAL_PROTEST tripled from 14.8 percent in 2023 to 46.2 percent in 2025"
  • Testimonial injustice: A type of epistemic injustice where a speaker’s credibility is unfairly deflated due to perceived identity. "Testimonial injustice in Fricker's original account requires that a speaker's credibility be deflated because of their perceived identity"
  • VADER (Valence Aware Dictionary and sEntiment Reasoner): A rule-based sentiment analysis tool designed for social media text. "the VADER (Valence Aware Dictionary and sEntiment Reasoner) sentiment compound score was computed for every LLM- validated REAL accusation on Reddit,"
  • Wilson confidence interval: A method for estimating binomial proportions with better small-sample properties than the normal approximation. "with Wilson 95% confidence intervals"
View Paper Prompt  View All Prompts 

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 3 tweets with 142 likes about this paper.

Sign Up for Free

HackerNews

  1. Accusations of 'AI Slop' Don't Screen for AI Text (4 points, 0 comments)