SHARP: Social Harm Analysis in AI Systems

• Social Harm Analysis via Risk Profiles (SHARP) is a distributionally-aware framework that defines social harm as a multivariate risk over dimensions like bias, fairness, ethics, and epistemic reliability.
• It integrates aspect-oriented hazard analysis, network-based profiling, and pathway modeling to quantify tail risks and capture rare, high-severity failures in sociotechnical systems.
• SHARP delivers actionable insights through risk report cards and metrics such as CVaR, aiding governance and dynamic intervention in advanced AI/ML deployments.

Social Harm Analysis via Risk Profiles (SHARP) is a multidimensional, distributionally-aware methodological framework for the systematic identification, modeling, quantification, and governance of social risks posed by advanced AI and ML systems. SHARP synthesizes aspect-oriented hazard analysis, network-based profiling, path modeling, quantitative and distributional risk metrics, and systems-engineering controls to produce rigorous, scenario-specific harm evaluations. It is motivated by the inadequacy of scalar risk scores in capturing the distributional structure, rare high-severity failures, and cross-dimensional dependencies characteristic of complex sociotechnical deployments (Abhishek et al., 29 Jan 2026).

1. Formal Definition and Methodological Taxonomy

SHARP formalizes social harm as a multivariate random variable over impacted dimensions, such as bias, fairness, ethics, and epistemic reliability. Let $H \in [0,1]^4 = (B, F, E, K)$ denote the per-prompt vector of normalized sub-index severities for a given model and prompt, where $B$ is bias, $F$ is fairness, $E$ is ethics, and $K$ is epistemic reliability (Abhishek et al., 29 Jan 2026).

The union-of-failures aggregation is defined as the probability of any-dimensional harm:

$$H^{any}_{M,q} = 1 - \prod_{i \in \{B,F,E,K\}} (1 - h_{i,M,q}),$$

with an additive cumulative log-risk reparametrization:

$$\ell_{i,M,q} = -\ln(1 - h_{i,M,q} + \varepsilon),\quad L_{M,q} = \sum_{i} \ell_{i,M,q}.$$

L is unbounded as any sub-index approaches 1, allowing tail differentiation beyond bounded risk measures.

SHARP proceeds via a structured pipeline:

1. Aspect-Oriented Hazard Analysis: Systematic generation of hazard scenarios using a first-principles decomposition of system aspects—Capabilities, Domain Knowledge, Affordances, Impact Domains (Wisakanto et al., 25 Apr 2025).
2. Risk Pathway Modeling: Construction of causal chains from source aspect to social impact, using both forward and backward chaining and scenario decomposition (event/fault trees) (Wisakanto et al., 25 Apr 2025).
3. Network-based Profiling: Multilayered network analysis of actor relations, derived topological metrics, and community structure to inform risk likelihoods for coordinated or relational harms (Colladon et al., 2021).
4. Quantitative and Distributional Metrics: Estimation at the pathway level via likelihood–severity products, and at the system/model level by risk-sensitive statistics such as Conditional Value-at-Risk (CVaR$_{0.95}$) over the log-risk distribution (Abhishek et al., 29 Jan 2026).
5. Uncertainty Management: Explicit scenario decomposition, reference scales (likelihood/severity levels), and evidence-tracing protocols capture epistemic uncertainty and calibrate model-team alignment (Wisakanto et al., 25 Apr 2025).
6. Aggregation and Governance: Synthesis in a risk report card with mean, volatility, VaR, CVaR, marginal sub-index exposures, and policy-actionable thresholds (Abhishek et al., 29 Jan 2026, Wisakanto et al., 25 Apr 2025).

2. Network-Based SHARP: Sociotechnical Risk Relationalization

In contexts where social harm propagates through relational ties—financial fraud, trafficking, coordinated disinformation—SHARP leverages multi-layer network analysis. Actors (nodes) and their financial, communicative, spatial, or organizational connections (edges) are mapped into one or more networks: transactions, sector-specific, geographical risk, tacit linkages (common beneficial owners), etc. Canonical metrics include in/out/all-degree, closeness, betweenness, structural constraint (Burt), and network motifs (Colladon et al., 2021).

Risk profiles are then constructed via predictive modeling (e.g., logistic regression, random forests, SVMs) using these metrics. The outcome variable is typically confirmed involvement in harmful activity, with risk flagged either individually or at the community/cluster level. Visual analytical overlays and dynamic dashboarding are common for practitioner use (Colladon et al., 2021).

This approach generalizes readily: relational mapping and network scoring are iterated across different harm modalities (e.g., radicalization, trafficking, online abuse), with domain-specific attributes and outcome labels (Colladon et al., 2021).

3. Pathway Modeling and Probabilistic Risk Quantification

SHARP emphasizes explicit modeling of causal chains from source system aspect to terminal harm, defined as a risk pathway. Each pathway decomposes into conditional steps:

1. Source Aspect (e.g. integrative planning)
2. Adjacent Hazard (e.g. bypass filter)
3. Intermediate Steps (e.g. exploit vulnerability)
4. Propagation Operator(s) (e.g. fragmentation, adversarial targeting)
5. Terminal Hazard (e.g. infrastructure collapse)
6. Impact Domain (e.g. ecological, societal, bodily)

The probability of the full pathway is

$P_\text{path} = \prod_{j=1}^n P_j,$

with $P_j$ as the likelihood of each conditional step. Severity is assigned on a harm scale (e.g., HSL 1–6). Path risk is $B$0, with system-level risk as $B$1 (Wisakanto et al., 25 Apr 2025).

For LLM-centric frameworks, per-prompt harm vectors are aggregated across prompts, and tail metrics (e.g., CVaR) inform governance and deployment (Abhishek et al., 29 Jan 2026).

4. Distributional and Tail-Focused Risk Profiling

Distributional tail analysis is central to SHARP’s critique of mean-centric benchmarking. For each model, the empirical distribution $B$2 (across evaluation prompts) is characterized by mean $B$3, volatility $B$4, VaR$B$5, and, critically, CVaR$B$6—the mean log-risk in the upper $B$7 tail. CVaR$B$8 typically captures the average severity for the worst 5% of evaluated scenarios, representing likely outcomes in catastrophic or high-impact failure regimes (Abhishek et al., 29 Jan 2026).

Marginal sub-index CVaRs (for bias, fairness, ethics, epistemic) and tail-attribution shares further inform which risk dimension dominates the extreme failure distribution—critical for targeted mitigation.

Empirically, models with near-identical mean risk differ by greater than twofold in CVaR$B$9, with tail exposure driven by specific sub-indices (e.g., bias for open models, epistemic for LLMs with adversarial prompt vulnerabilities) (Abhishek et al., 29 Jan 2026).

5. Uncertainty Management, Documentation, and Governance Integration

For scenarios with limited historical data or high novelty, SHARP mandates explicit decomposition of uncertainty via:

• Event/fault-tree scenario breakdown, isolating branches with maximal epistemic uncertainty.
• Reference scales for likelihood (LL-0 to LL-8, logarithmic) and severity (HSL-1 to HSL-6, metric-anchored).
• Evidence-tracing logs for each risk entry, documenting data sources, assumptions, rationale for level assignments, and assessor roles/timestamps.
• Team recalibration protocols using anonymized rationales and maximally-defensible risk level selection (Delphi-inspired) to combat underestimation bias (Wisakanto et al., 25 Apr 2025).

The aggregation phase produces a risk report card, mapping each assessed risk to a discrete risk level (e.g., RL 0–9), with system-level absolute risk, scenario buckets, and RL-heatmap visualizations. Governance interfaces are specified: RL thresholds map directly to required actions (e.g., RL >= 7: halt deployment; RL 4-6: controls required; RL <= 3: standard monitoring) (Wisakanto et al., 25 Apr 2025).

6. Applications, Case Studies, and Experimental Results

LLM Harm Profiling

Application of SHARP to eleven leading LLMs on 901 prompts reveals substantial distributional heterogeneity: for instance, gpt-4o has mean log-risk 0.748 and CVaR$F$0 4.519, compared to claude-sonnet-4.5 with 0.158 and 1.689, respectively. Tail attribution analysis identifies bias as the dominant contributor to extreme risk in some models, epistemic unreliability in others (Abhishek et al., 29 Jan 2026).

Sociotechnical Case Analysis

STPA-extended SHARP identifies hazards, losses, unsafe control actions, and causal pathways in regulated system contexts, e.g., the ML-driven Prescription Drug Monitoring Program (PDMP) case. It produces granular hazard–loss matrices, risk profiles, and design-level control requirements, including fairness-driven objectives, protected-group calibration, differential auditing, role-based access, and override interfaces (III et al., 2022).

Real-Time Behavioral and Networked Risk Interventions

For domains such as mental health risk detection or financial crime prevention, SHARP incorporates profile construction via sequence modeling, BERT-based classification, or multilayer network embedding, with evaluation using early risk detection error ($F$1), macro-F1, and latency-weighted scores (Bucur et al., 2021, Soldaini et al., 2018, Colladon et al., 2021). SHARP-enabling pipelines support both “always-on” intervention dashboards and retrospective auditing.

7. Practical Recommendations and Implementation Considerations

• Use a fixed, scenario-relevant prompt/test corpus to standardize risk profile comparability.
• Employ ensemble LLM-as-judge or expert scoring (log-sum-exp aggregation) for prompt-level harm indexing.
• Prefer CVaR- and volatility-based thresholds for deployment gating, supplementing mean scores.
• Disaggregate risk across dimensions and monitor marginal and joint tail exposures.
• Maintain full scenario documentation, evidence chains, and risk-calibration governance.
• Update risk thresholds and model selection via participatory processes, ensuring that interventions addressing one dimension do not introduce new vulnerabilities elsewhere (Abhishek et al., 29 Jan 2026, Wisakanto et al., 25 Apr 2025, III et al., 2022).

SHARP thus provides a comprehensive, evidence-traceable, multidimensional approach to social harm evaluation, management, and governance for high-stakes AI/ML deployments. It operationalizes advanced risk modeling, distributional statistics, and systems engineering methods for rigorous, actionable harm profiling.