Hierarchical Safety Taxonomy

• Hierarchical safety taxonomies are explicitly layered frameworks that categorize, trace, and mitigate risks in complex technical and sociotechnical systems.
• They integrate top-down regulatory analysis, bottom-up data mining, and expert reviews to ensure granular and comprehensive risk assessment.
• Applications include streamlined audit processes, enhanced compliance, and improved model training that boost rare-event detection and safety metrics.

A hierarchical safety taxonomy is an explicitly layered framework for categorizing, analyzing, and operationalizing the identification and mitigation of safety risks in complex technical and sociotechnical systems. These taxonomies are foundational for ensuring robust, transparent, and fine-grained risk management in domains such as AI alignment, autonomous vehicles, industrial safety, and regulatory benchmarking. Hierarchical structuring enables the partitioning of risks from abstract domains through successively finer categories to atomic, testable leaves, affords traceability of failures or hazards to their sources, and supports both coarse- and fine-grained safety auditing and intervention.

1. Formal Structure of Hierarchical Safety Taxonomies

Hierarchical safety taxonomies are tree- or DAG-structured systems where each risk or hazard category is assigned a unique position within at least two or more explicit, strictly nested levels. The formal structure, cardinality at each tier, and parent-child mappings provide complete traceability and mutual exclusivity among risk categories.

A canonical example is the four-level taxonomy from AIR-Bench 2024, mapping 8 government regulations and 16 corporate policies into a strict containment hierarchy:

• Tier 1 (C₁): Top-level domains (4, e.g. Operational, Content Safety, User Rights, Societal Harms)
• Tier 2 (C₂): Mid-level groupings (16, e.g. Security Risks, Privacy, Hate/Toxicity)
• Tier 3 (C₃): Sub-domains (45, e.g. Hate Speech, Political Influence)
• Tier 4 (C₄): Terminal, fully specified risks (314, each corresponding to explicit regulatory or policy-defined scenarios)

Formally: $$C_i = \left\{c(i,1),c(i,2),\ldots,c(i,|C_i|)\right\}, \quad i=1,2,3,4$$ Each $c\in C_i$ for $i>1$ has one parent in $C_{i-1}$, enforced by a function: $\text{parent}\colon C_i\to C_{i-1}$ This results in a strict, non-overlapping partition of the risk universe, enabling full upward and downward traversal within the taxonomy (Zeng et al., 2024).

Other taxonomies, such as the three-level model in aviation safety event extraction (Occurrence → Subject), or the four-level risk decomposition in DriveSafe for LLM automotive assistants, mirror this systematic nesting for granular identification and role assignment (Zhao et al., 2024, Kumar et al., 17 Jan 2026).

2. Core Dimensions and Levels in Domain-Specific Taxonomies

The precise selection of levels and their semantic scope differ by application context, with each level providing increasing refinement and operational relevance.

General content safety example (Aegis2.0):

• Safe / Needs Caution: Non-hazard, ambiguous cases
• 12 Core Hazard Categories: (e.g., Hate, Suicide, PII/Privacy, Criminal Planning, etc.)
• 9 Fine-Grained Subcategories: (e.g., Illegal Activity, Fraud/Deception, Malware) (Ghosh et al., 15 Jan 2025)

Automotive/LLM driving assistant safety (DriveSafe):

• Level 1 (Domains): Technical, Business, Societal, Ethical Risks
• Level 2 (Categories): E.g., Legal Risks, Embedded Risks, Bias & Stereotype Risks
• Level 3 (Failure Modes): E.g., Compliance Drift, Misuse of System
• Level 4 (Atomic Risks): E.g., Incorrect Speed-Limit Interpretation (Kumar et al., 17 Jan 2026)

Self-adaptive system adaptation safety (Diemert et al., 2022):

• Type 0: Non-Interference
• Type I: Static Assurance (fully defined and pre-verified options)
• Type II: Constrained Assurance (option-specific constraints, dynamic safety case update)
• Type III: Dynamic Assurance (open-ended, run-time proof required)

Hierarchical depth is thus tuned to the domain’s operational and regulatory needs, balancing sufficiency of coverage with auditability and practical implementation.

3. Methodologies for Construction and Validation

Rigorous hierarchical safety taxonomies are derived using a combination of top-down regulatory analysis, empirical itemization, and bottom-up data-driven discovery. Exemplified methodologies include:

• Direct grounding in standards/policies: AIR-Bench 2024 maps policy statements from regulatory documents into taxonomy leaves, ensuring completeness relative to societal requirements (Zeng et al., 2024).
• Hybrid annotation/data mining: Aegis2.0 standardizes post-hoc subcategories emerging from open “Other” annotations, supplementing prescriptive lists (Ghosh et al., 15 Jan 2025).
• Expert panels and iterative review: DriveSafe convened domain experts and conducted multi-round workshops to finalize scenario mappings and ensure realism (Kumar et al., 17 Jan 2026).
• Hierarchical modeling for classification: In aviation event taxonomy extraction, hierarchical attention, recursive regularization, and KL-based label-distribution penalties encode the taxonomy structure directly into the learning process (Zhao et al., 2024).
• Causal linkage and propagation: HAZOP-like analyses (e.g., HILLS) stratify system, ML lifecycle, and inner model levels, propagating hazards upward via both qualitative guideword matching and Bayesian network inference (Qi et al., 2022).

Taxonomy validation typically includes inter-annotator agreement measurement (e.g., Cohen’s κ=0.80 in DriveSafe), exhaustive coverage assessment (as in AIR-Bench’s 314-leaf alignment), and benchmarking against established detection/guard models.

4. Applications and Evaluation Metrics

Hierarchical safety taxonomies are foundational to:

• Benchmarking and evaluation: Structured scenario generation and scoring, e.g., refusal rates per risk in LLM safety evaluation (DriveSafe, SALAD-Bench, AIR-Bench) (Kumar et al., 17 Jan 2026, Li et al., 2024, Zeng et al., 2024).
• Guard model training: Parameter-efficient fine-tuning using taxonomy-annotated corpora (Aegis2.0), enabling topic generalization and evolving guardrails (Ghosh et al., 15 Jan 2025).
• Systematic safety analysis: HAZOP analogues for ML-enabled systems distinguish hazard sources across architectural, lifecycle, and functional levels, enabling targeted mitigations at each point in the propagation chain (Qi et al., 2022).
• Domain-specific adaptation: Aviation, automotive, and Chinese LLM safety benchmarks instantiate and localize taxonomic constructs for their respective contexts, with performance metrics tailored accordingly (e.g., accuracy, rejection rate, responsibility rate, harmfulness rate, macro-F1 on rare labels) (Zhao et al., 2024, Zhang et al., 2024, Ghosh et al., 15 Jan 2025).
• Regulatory compliance: Taxonomies serve as blueprints for auditing conformity to binding standards, supporting both aggregate and drill-down compliance assessments (e.g., AIR-Bench at Tier 1–4 granularity) (Zeng et al., 2024).

5. Comparative Analysis and Impact

The adoption of hierarchical, regulation-grounded safety taxonomies marks a decisive improvement over flat or ad-hoc risk lists. Key advantages:

• Completeness: By construction, taxonomies such as AIR 2024 achieve complete coverage of all terminal risks recognized by authoritative sources; empirical benchmarks like SALAD-Bench capture only ~71% of level-3 AIR risks (Zeng et al., 2024).
• Hierarchical clarity: Strict, nested containment allows aggregate and fine-grained metrics, prevents category overlap/gaps, and facilitates robust scenario generation (Kumar et al., 17 Jan 2026, Ghosh et al., 15 Jan 2025).
• Testability: Leaf categories correspond precisely to actionable, evaluable scenarios, supporting transparent metric calculation and targeted mitigations (Kumar et al., 17 Jan 2026, Qi et al., 2022).
• Regulatory alignment: Direct mapping to regulations and policies ensures relevance and audit traceability (Zeng et al., 2024).
• Model improvement: Integration into supervised and hybrid classification frameworks significantly enhances rare-event detection (reported +19–24.8% macro-F1 gains in aviation applications) and sensitivity to context/long-tail hazards (Zhao et al., 2024, Sun et al., 2021).

6. Limitations and Ongoing Developments

While hierarchical safety taxonomies offer superior granularity and comprehensiveness, several challenges remain:

• Taxonomy drift and evolution: Emergent risks may not be immediately present in source regulatory texts or annotations, necessitating periodic revision and extension (as seen with Aegis2.0’s post-hoc fine-grained categories).
• International and domain localization: Benchmarks such as CHiSafetyBench adapt frameworks for local regulatory requirements and language/cultural context, underscoring the need for region-specific taxonomies with cross-mapping for global model evaluation (Zhang et al., 2024).
• Empirical bottlenecks: Annotation validity, inter-annotator agreement, and scenario coverage all affect practical utility and require substantial expert resources (Kumar et al., 17 Jan 2026).
• Context-sensitivity: Explicit modeling of context-dependent unsafety, as developed in dialogue safety taxonomies, remains challenging for hierarchical systems and requires sophisticated context-aware detection architectures (Sun et al., 2021).

A plausible implication is the ongoing trend toward hybrid approaches that incorporate policy-driven, expert-reviewed hierarchies with data-mined augmentation and context-sensitive extensions. This enables both static and dynamic assurance of safety in increasingly complex, adaptive sociotechnical systems.