Why AI Harms Can't Be Fixed One Identity at a Time: What 5300 Incident Reports Reveal About Intersectionality

Abstract: AI risk assessment is the primary tool for identifying harms caused by AI systems. These include intersectional harms, which arise from the interaction between identity categories (e.g., class and skin tone) and which do not occur, or occur differently, when those categories are considered separately. Yet existing AI risk assessments are still built around isolated identity categories, and when intersections are considered, they focus almost exclusively on race and gender. Drawing on a large-scale analysis of documented AI incidents, we show that AI harms do not occur one identity category at a time. Using a structured rubric applied with a LLM, we analyze 5,300 reports from 1,200 documented incidents in the AI Incident Database, the most curated source of incident data. From these reports, we identify 1,513 harmed subjects and their associated identity categories, achieving 98% accuracy. At the level of individual categories, we find that age and political identity appear in documented AI harms at rates comparable to race and gender. At the level of intersecting categories, harm is amplified up to three times at specific intersections: adolescent girls, lower-class people of color, and upper-class political elites. We argue that intersectionality should be a core component of AI risk assessment to more accurately capture how harms are produced and distributed across social groups.

• The paper introduces a novel rubric for quantifying intersectional harms in AI using 5,300 incident reports, emphasizing counterfactual causal relevance.
• It reveals that non-traditional identity markers such as age and political identity are as significant as race and gender, challenging conventional fairness assumptions.
• Results indicate amplified harms at key identity intersections, underscoring the need for intersectionality-focused risk assessment and governance in AI.

Intersectional Harms in AI: Empirical Evidence from 5,300 Incident Reports

Methodological Innovations and Scope

This study systematically analyzes intersectional harms in AI using 5,300 reports from 1,200 incidents in the AI Incident Database. The methodology leverages a structured rubric for annotating harmed subjects, identity categories, and harm mechanisms, operationalized via LLMs and validated for annotation reliability. Key innovations include explicit counterfactual causal relevance assessment and rigorous deduplication across subjects and incidents, ensuring incidence counts reflect causal identity linkages rather than mere report mentions.

Figure 1: The four-step methodology for incident collection, identity extraction, counterfactual relevance assessment, and intersectional harm quantification.

Empirical Patterns in Identity-Based AI Harms

Analysis reveals that age (32%) and political identity (27%) occur as frequently as race (25%) and gender (24%) in documented, causally relevant AI harms. This contradicts the prevailing assumption in fairness research that race and gender are uniquely dominant axes for algorithmic risk. Within identity categories, structural vulnerability rather than numerical majority determines harm exposure: adolescents and children are more frequently targeted than adults (Figure 2); political elites, activists, and voters comprise the majority of political identity harms, while ideologically defined groups are far less exposed; people of color and females are overwhelmingly more likely to be harmed relative to their counterparts.

Figure 2: Example extraction from an incident report, highlighting identity categories, values, and harm mechanisms for three high-profile female subjects.

Figure 3: Category-level prevalence of causally relevant identity attributes in incidents, demonstrating equal prominence for age and political identity relative to race and gender.

Figure 4: Value-level prevalence within top identity categories, showing amplified harm for adolescents, political elites, lower class, people of color, U.S. nationality, and females.

Six harm mechanisms are observed for single-category incidents: sexualizing (deepfake attacks primarily on females/adolescents), steering (algorithmic recommendation toward toxic/extremist content for males), matching (biometric misrecognition disproportionately affecting people of color), inferring (nonconsensual attribute prediction, notably for sexuality), gating (binary verification excluding gender-diverse subjects), and manipulating (identity-driven political disinformation).

Intersections and Harm Amplification

Intersectional analysis identifies three dominant axes: nationality + political identity, age + gender, nationality + class, each associated with sharp amplification of harm exposure. Statistical amplification scores demonstrate that intersections of female gender and adolescent age, lower class and people of color, and upper class and political elite recur two to three times more frequently than expected under independence.

Figure 5: Intersection heatmap of identity categories, highlighting concentration of harm across specific pairings such as nationality/political identity, age/gender, and nationality/class.

Figure 6: Amplification scores between most prevalent identity intersections, showing strong effects for adolescent girls, lower class people of color, and upper-class political elites.

Intersectional harm is not simply additive: compounded vulnerability is evident for marginalized groups (e.g., adolescent girls, low-income minorities), but elite visibility also introduces amplified risk through targeted disinformation, impersonation, or reputational attacks for public figures and political leaders.

Institutional and Social Power Dynamics

The study identifies five recurrent intersectional harm themes tied to sociotechnical systems and institutional logics:

• Algorithmic suspicion/criminalization: Disproportionate risk scoring, fraud detection, and intervention in welfare and immigration domains for low-income, racialized, dual-nationality subjects.
• Sexualized exploitation: Generative image/media technologies weaponized to target adolescent girls and women from marginalized groups, resulting in deepfakes, “nudification,” invasive profiling, and psychological harm.
• Environmental violence: AI-enabled infrastructure disproportionately concentrates pollution and resource burden in racialized/lower-class neighborhoods with children.
• Political manipulation: AI-generated media drives targeted political disinformation against young voters and minority communities, exploiting intersectional vulnerabilities in democratic contexts.
• Militarized violence: Automated targeting systems in conflict zones disproportionately classify and attack intersecting groups defined by nationality, race, and political identity, resulting in mass civilian harm.

Annotation Quality and Limitations

Validation shows high annotation reliability for harmed subject identification and causal relevance assignment (98% and 92% accuracy, respectively). The largest misattribution is gender, mostly due to LLM reluctance to infer from implicit cues, whereas human annotators are prone to broader inference (Figure 7).

Figure 7: Maximum misattribution rate for identity categories, highlighting high rates for gender due to implicit cues and conservative LLM extraction.

The incident database suffers from U.S.-centric, media-driven bias, with underrepresentation of harms for less visible groups and contexts. The analysis focuses on direct victims; indirect, ripple effects are rarely captured. Global and institutional diversity is limited. LLM-based annotation, though validated, may embed both rubric-driven and model-driven interpretive bias.

Implications for AI Risk Assessment and Governance

The findings mandate a fundamental redirection in AI risk assessment: single-axis fairness categories are insufficient; intersectionality must be central for empirical harm anticipation, audit, and mitigation. Intersectional rubrics should be adopted at all phases of the AI lifecycle:

• Design: Move beyond typical user personas; stress-test intersectional exposure likely in each domain.
• Deployment/audit: Regulatory scrutiny must target amplified risk at observed intersections, requesting intersection-specific mitigation and documentation.
• Monitoring: End-user participatory audits should leverage intersectional harm detection and reporting; severity and cumulative impacts must be assessed alongside incidence.

Empirical incident analysis should inform technical fairness metrics, procurement standards, and regulatory action, including in future legislative frameworks inspired by EU AI Act and similar regulatory models.

Future Directions

Addressing reporting bias requires integration of regulatory filings, whistleblower reports, and incident data from underrepresented regions and populations. Frameworks should be extended for secondary and indirect victim documentation, multisource validation, and alternative annotation strategies beyond LLMs.

Conclusion

Intersectionality is empirically required for accurate AI risk assessment and governance. Harms are neither isolated nor additive across single identity categories; amplification at key intersections reflects compounding vulnerability and institutional visibility alike. Responsible AI research and practice must broaden beyond race and gender and address intersectional patterns most frequently harmed in real deployments. The rubric and dataset released by this work provide operational tools and analytical benchmarks for future intersectionality-informed AI risk assessment and mitigation (2604.24519).