DeepSTPA: Hazard Analysis for ML Systems

• DeepSTPA is a methodological extension of STPA designed to analyze hazards in machine learning systems by addressing vulnerabilities in both data processes and internal model structures.
• It expands traditional hazard analysis through lifecycle stage modeling and layerwise decomposition, enabling detection of unsafe control actions that conventional methods might miss.
• The approach, validated through case studies like DRL-based AEB, offers enhanced causality tracing and safety assurance, albeit with increased complexity and modeling effort.

DeepSTPA is a methodological extension of Leveson's Systems Theoretic Process Analysis (STPA) for hazard analysis, tailored to the demands of Learning-Enabled Systems (LESs). Designed to address hazards whose origins reside in both the data-driven development process and the internal structure of ML components, DeepSTPA systematically expands the scope of analysis beyond the traditional boundaries of component-level operation. By formalizing interconnected control loops across all ML lifecycle stages and each layer within the ML model architecture, DeepSTPA enables comprehensive identification of unsafe control actions (UCAs) and their root causes—many of which elude conventional STPA approaches (Qi et al., 2023).

1. Rationale and Conceptual Foundation

Leveson's original STPA provides a systematic approach to hazard analysis, grounded in modeling control loops for each control component (Controller → Control-Action → Process → Feedback → Controller). However, in the context of machine learning–intensive or software-defined control, STPA's abstraction treats the ML subsystem as a monolithic black box and restricts analysis to the operational phase. DeepSTPA relaxes these constraints through two principal innovations:

• Extension of STPA control loop modeling to each stage of the ML component's lifecycle: environment/data preparation, training, evaluation, deployment, and operation.
• Internal decomposition of the ML model, where each architectural layer (e.g., convolutional, pooling, fully-connected, output) is mapped to its own control loop, exposing internal state and action granularity.

These enhancements address hazards arising from data-collection inadequacies, model architecture choices, training regime flaws, and layer-specific misparameterizations—sources of risk not accessible to component-boundary analyses.

2. Formal Extension of Control Loops to the ML Lifecycle

In standard STPA, one control loop per component suffices. DeepSTPA generalizes this to a cascade of control loops across the major stages of the ML lifecycle. Let $$S = \{\mathrm{EnvPrep}, \mathrm{ModelTrain}, \mathrm{Eval}, \mathrm{Deploy}, \mathrm{Operate}\}$$ denote the stages. For each stage $s \in S$, DeepSTPA defines a loop:

$CL_s = (C_s, P_s, A_s, M_s, F_s)$

where:

• $C_s$: Stage-specific controller (e.g., developer, training algorithm, verification & validation team)
• $P_s$: Process or subcomponent controlled in $s$ (e.g., simulation environment, model trainer)
• $A_s$: Control actions (dataset specification, hyperparameter selection, deployment triggers)
• $M_s$: Process model (e.g., developer’s abstraction of data generalization, Q-value update dynamics)
• $F_s$: Feedback (accuracy metrics, reward signals, test coverage)

These loops are interconnected, passing control actions and artifacts as inputs to the subsequent stage (e.g., simulation scenarios feed into model training; trained weights become the evaluation subject). The union of these forms:

$$CL_{\mathrm{Deep}} = \bigcup_{s\in S} \left\{\,C_s \rightarrow A_s \rightarrow P_s \rightarrow F_s \rightarrow C_s\,\right\}$$

This formalization explicitly models influences and potential failure points introduced at any stage, including pre-operational phases.

3. Layer-wise Modeling of ML Internals

DeepSTPA departs from black-box assumptions by decomposing the ML model into $s \in S$0 explicit layer-level loops:

$s \in S$1

Here,

• $s \in S$2: Parameter updater for layer $s \in S$3 (e.g., sets $s \in S$4 $s \in S$5, stride $s \in S$6)
• $s \in S$7: Layer-specific action (e.g., apply convolution with given parameters)
• $s \in S$8: Resulting process (e.g., feature map after convolution)
• $s \in S$9: Model or assumption (e.g., receptive field coverage)
• $CL_s = (C_s, P_s, A_s, M_s, F_s)$0: Feedback (e.g., statistics of feature-map activations, gradient flow)

Control actions and safety constraints are now defined at the layer and parameter level. For example, in a convolutional layer: $CL_s = (C_s, P_s, A_s, M_s, F_s)$1 An unsafe control action is summarized as: $CL_s = (C_s, P_s, A_s, M_s, F_s)$2 where $CL_s = (C_s, P_s, A_s, M_s, F_s)$3 characterizes feature density, and $CL_s = (C_s, P_s, A_s, M_s, F_s)$4 are specification bounds.

4. Formal Representation and Analytical Artifacts

The DeepSTPA framework utilizes extended block diagrams that preserve the STPA notation (rectangles for controllers/processes, directed edges for actions and feedback), but multiplies instances both horizontally (lifecycle stages) and vertically (layerwise). State variables in DeepSTPA encompass not only classical process state but also:

• Dataset quality and diversity metrics
• Reward function parameters for RL
• Hyperparameters (learning rate, buffer size)
• Neural network weights/activation distributions
• Q-value statistics for value-based RL agents

Safety constraints articulate requirements over these expanded domains (e.g., ensuring that "reward function penalizes collisions above $CL_s = (C_s, P_s, A_s, M_s, F_s)$5 m/s²", or "convolutional stride does not exceed specifications to preserve small obstacle features"). This layered representation supports detection of unsafe actions (UCAs) and the tracing of causal scenarios specific to ML domain phenomena (e.g., distributional drift, non-convergent training, poor generalization).

5. Comparative Case Study: DRL-based Autonomous Emergency Braking (AEB)

A case study on a Deep Reinforcement Learning–based Autonomous Emergency Braking (DRL-AEB) system illustrates the comparative coverage of DeepSTPA versus baseline STPA [Sun et al. 2021, (Qi et al., 2023)]. Standard STPA models a single operational loop (Controller → AEB algorithm → Vehicle dynamics → Sensors → Controller), identifying hazards such as missing or late brake signals and issuing boundary-level safety requirements.

In contrast, DeepSTPA instantiates:

• Five lifecycle-stage loops: Environment Preparation, Model Training, Evaluation, Deployment, Operation
• Multiple layer-wise loops for internal DQN layers: convolutional, pooling, fully-connected, argmax

This expansion enabled DeepSTPA to uncover hazard scenarios and unsafe control actions unobservable to classical STPA analysis, including:

• UCA-1: Incomplete reward design leads RL agent to neglect braking in rare cases
• UCA-2: Incorrect hyperparameterization (learning rate, replay buffer size) undermines policy convergence
• UCA-4: Oversized convolutional kernel/stride removes small obstacle detection
• UCA-5: Imperfect $CL_s = (C_s, P_s, A_s, M_s, F_s)$6 implementation leads to suboptimal braking choices

Associated causal scenarios (insufficient generalization, exploration/exploitation imbalance, sparse rewards) and derived safety requirements (data augmentation, $CL_s = (C_s, P_s, A_s, M_s, F_s)$7-greedy exploration, experience replay, reward shaping) were similarly more comprehensive.

Comparison Table

Attribute	STPA	DeepSTPA
Identify hazards	partial	comprehensive
Identify causal factors	partial	comprehensive (ML-specific)
Lifecycle coverage	operation	development & operation
Analysis effort (time/cost)	low	higher
Method complexity	low	higher

DeepSTPA identified all the UCAs found by STPA and additional ones tied to ML training and architecture, providing a stricter and more detailed safety argument. The methodological trade-off is increased modeling and analytical burden (10–15 loops versus one), requiring greater domain expertise and resources.

6. Methodological Steps and Scope Expansion

DeepSTPA preserves the high-level STPA procedural structure—(1) Define mishaps, (2) Model control loops, (3) Identify UCAs, (4) Develop causal scenarios, (5) Create safety requirements—but significantly expands step (2) both horizontally (ML lifecycle stages) and vertically (layerwise decomposition). This dual-axis modeling grants DeepSTPA the analytical fidelity required to uncover hazards and root causes rooted in data practices, training artifacts, and network architecture, which would remain opaque under traditional approaches.

7. Context and Implications for Learning-Enabled Safety

DeepSTPA is positioned as a response to the growing prevalence of ML, particularly deep learning, in safety-critical domains where latent hazards are not only operational but also epistemic, stemming from choices and errors throughout the data/model development pipeline. Its explicit control-loop formalism at both lifecycle and architectural levels establishes a systematic basis for ML-aware hazard analysis.

This suggests that DeepSTPA's adoption can generate richer, end-to-end traceability in safety cases for LESs, surfacing issues otherwise lost at the component interface. However, the higher analytical complexity and resource requirements set a practical barrier, indicating an inherent trade-off between depth of hazard coverage and process manageability (Qi et al., 2023).