Epistemic Injustice in Language Models: An Audit of Pretraining Filters and Guardrails

Abstract: Modern LLMs rely on pretraining filters to remove undesirable content from training corpora and inference-time guardrails to suppress undesirable outputs during deployment. In this paper, we examine how these filtering and moderation decisions produce forms of epistemic erasure and reveal tensions both across automated systems and between these systems and human judgment. We audit four pretraining filters and three inference-time guardrails on Common Crawl sentences containing gender and regional-origin mentions, together with a manually annotated subset of 500 sentences. Our analysis shows that filtering and guardrail decisions are strongly associated with blocklist-based lexical cues, while frequently failing to flag content containing private information or explicit hate speech. At the same time, marginalized groups, particularly transgender people, women, and Central Americans, are significantly over-flagged across systems. Human annotators, by contrast, would retain 88.5\% of filter-flagged and 91.3\% of guardrail-flagged content, often recognizing representational harms arising from tensions of content removal that current systems fail to capture. Taken together, our findings document a form of epistemic erasure in which mentions of marginalized groups are disproportionately removed before pretraining and additionally suppressed again at inference time.

• The paper demonstrates that pretraining filters and guardrails lead to systematic epistemic erasure, particularly affecting transgender, female, and Central American identities.
• The study employs quantitative methods, triangulation, and manual annotation to reveal low agreement among diverse filtering and guardrail systems.
• The audit exposes a significant gap between automated flagging and human judgment, calling for community-reflexive design in LLM safety mechanisms.

Epistemic Injustice in LLMs: A Formal Analysis of Pretraining Filters and Guardrails

Introduction

"Epistemic Injustice in LLMs: An Audit of Pretraining Filters and Guardrails" (2606.05936) delivers a methodologically rigorous and systematic audit of both pretraining data filters and inference-time guardrail models in widely adopted LLM pipelines. The study interrogates whose identities and perspectives are disproportionately subject to suppression or flagging, with a particular focus on gender and regional-origin mentions. The core finding is that both filtering and moderation systems do not merely target toxicity or unsafe content syntactically, but in practice shape the representational boundaries of deployed models, disproportionately erasing content linked to marginalized groups. This is not a uniform effect: it interacts significantly with group identity, system taxonomy, and subjective human judgment. The analysis is technically grounded, featuring triangulation across several filtering and guardrail pipelines, large-scale Common Crawl sampling, and corroborative manual annotation.

Empirical Methodology

The audit framework considers four widely used pretraining filters (lexicon-based and classifier-based) and three inference-time guardrails, all deployed on 72,659 sentences sampled from English Common Crawl containing explicit gender or region-associated mentions. Identity extraction is performed using NER alongside curated Wikidata lexicons. Analytical splits separate filter-flagged and unflagged sentences, supporting differential analysis. The framework leverages both automated system outputs and a subset of 500 sentences manually annotated by human raters, who are asked to specifically weigh the harms of both retaining and removing flagged content.

Agreements and Divergences Between Filters and Guardrails

The agreement analysis, quantified using pairwise Cohen's kappa, reveals several key patterns: agreements are generally low across systems ostensibly designed for similar purposes (harm reduction), indicating divergent operationalizations of "toxicity" and "harm." Notably, the lexicon-based Shutterstock filter yields moderate-to-strong agreement with all three guardrails, whereas classifier-based filters (Hatebase, DOLMA, HateBERT) align only weakly. This outcome validates prior observations that blocklist-based filters often suppress broad swathes of non-normative identity content irrespective of context.

Figure 1: Pairwise Cohen's kappa between filter and guardrails, revealing variable system alignment, with bold and underlined distinctions reflecting statistical significance.

Guardrails themselves manifest distinct biases according to their design provenance: MD-Judge and Qwen3Guard show substantial agreement, clustering apart from Llama-Guard, which emphasizes intellectual property and privacy dimensions more than group representation or toxicity.

Group-Dependent Over-Flagging and Epistemic Erasure

The identity-conditional analysis exposes robust and statistically significant over-flagging of marginalized group mentions. Mentions of transgender people are flagged at rates 1.5-1.8x higher than cis mentions across all three guardrails and Shutterstock. Women's mentions and those of non-binary or gender-nonconforming individuals also see a marked elevation. Central American regional mentions are among the most suppressed, with up to 99.3% flagged by Shutterstock, and nearly equivalent rates from Llama-Guard and Qwen3Guard. In contrast, Western European, Western Asian, and Northern European mentions are systematically under-flagged by guardrails, demonstrating a homogenizing effect that privileges certain geographies and identities.

Figure 2: Identity-dependent flag rates for gender modality and category, with highly significant over-flagging of transgender and non-binary categories.

Figure 3: Flag rates by regional category, emphasizing the over-flagging of Central America and differential behavior across regions.

Classifier-based filters such as Hatebase and HateBERT do not show these same group-specific patterns, due to the limitations of their lexical coverage. The effect sizes and $q$-values in the analyses reinforce that the epistemic erasure is systematic and not a stochastic or dataset-local artifact.

Human Annotation and System False Positives

Manual audit shows that human annotators sharply diverge from the system-mediated judgment: they would retain 88.5% of content flagged by filters and 91.3% of content flagged by guardrails as harmful. This finding is replicated across system types and is robust to random sampling. The qualitative coding of annotator rationales identifies practical and representational harms that are not captured by algorithmic systems, such as occupational visibility for sex workers, authentic group representation, and informational utility of contested text.

Figure 4: Human annotator retention rates for system-flagged content, highlighting systematic disagreement with automated verdicts.

The tension is clear: automated filters and guardrails prioritize removal from a harm-minimization perspective, but lack contextual or community-specific understanding, resulting in substantial epistemic loss—especially for already underrepresented or misrepresented groups.

Implications and Technical Recommendations

This audit demonstrates that both pretraining filters and guardrails operationalize harm in a manner that is systematically misaligned with human assessment and deeply skewed against marginalized identities. Documentation of filter and guardrail behavior by group, as advocated by Lucy et al. [lucy-etal-2024-aboutme], and community consultation on acceptable filtering regimes, are positioned as critical mitigations. Importantly, the risk of epistemic erasure—where model outputs invisibilize certain perspectives or entrench stereotypes—is compounded as models propagate biases present in both data and moderation infrastructure.

Technically, this points to the limitations of current blocklist and classifier architectures and harm taxonomies that are both underspecified and insufficiently reflexive. The design of future LLM safety mechanisms must shift toward pluralistic alignment and epistemic justice considerations, as highlighted by Sorensen et al. [sorensen2024roadmappluralisticalignment], including evaluation schemes incorporating affected demographic annotation and adversarial red-teaming [ganguli2022redteaminglanguagemodels].

Conclusion

This paper provides a comprehensive, quantitative, and qualitative audit of content moderation and data filtering in LLM pipelines. The results make a strong case that status quo harm reduction infrastructures—regardless of underlying taxonomy or geographic provenance—systematically over-flag and erase content about transgender people, women, Central Americans, and other marginalized identities. These patterns are not only isolated from human ethical judgment but are likely to have downstream impact on model capabilities, fairness, and the epistemic landscape of AI deployments. The study prescribes a shift towards explicit, empirically documented, and community-reflexive approaches in filter and guardrail design. Extensions to intersectional, religious, and political identities, as well as a broader annotation pool, constitute logical avenues for future work.

References:

• Stranisci et al., "Epistemic Injustice in LLMs: An Audit of Pretraining Filters and Guardrails" (2606.05936)
• Lucy et al., "AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters" [lucy-etal-2024-aboutme]
• Sorensen et al., "A Roadmap to Pluralistic Alignment" [sorensen2024roadmappluralisticalignment]
• Ganguli et al., "Red Teaming LLMs to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned" [ganguli2022redteaminglanguagemodels]