ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems (2102.09419v2)

Abstract: Autonomous CPSs are often required to handle uncertainties and self-manage the system operation in response to problems and increasing risk in the operating paradigm. This risk may arise due to distribution shifts, environmental context, or failure of software or hardware components. Traditional techniques for risk assessment focus on design-time techniques such as hazard analysis, risk reduction, and assurance cases among others. However, these static, design-time techniques do not consider the dynamic contexts and failures the systems face at runtime. We hypothesize that this requires a dynamic assurance approach that computes the likelihood of unsafe conditions or system failures considering the safety requirements, assumptions made at design time, past failures in a given operating context, and the likelihood of system component failures. We introduce the ReSonAte dynamic risk estimation framework for autonomous systems. ReSonAte reasons over Bow-Tie Diagrams (BTDs) which capture information about hazard propagation paths and control strategies. Our innovation is the extension of the BTD formalism with attributes for modeling the conditional relationships with the state of the system and environment. We also describe a technique for estimating these conditional relationships and equations for estimating risk based on the state of the system and environment. To help with this process, we provide a scenario modeling procedure that can use the prior distributions of the scenes and threat conditions to generate the data required for estimating the conditional relationships. To improve scalability and reduce the amount of data required, this process considers each control strategy in isolation and composes several single-variate distributions into one complete multi-variate distribution for the control strategy in question.

• The paper presents ReSonAte, a framework extending the Bow-Tie Diagram formalism to dynamically estimate risks in autonomous systems using runtime data.
• Evaluation via AGV and UUV simulations demonstrated ReSonAte's strong correlation between dynamic risk estimates and actual collisions, with efficient runtime computation (0.3ms).
• ReSonAte advances risk management for autonomous Cyber-Physical Systems by moving beyond static methods and enabling continuous improvement through runtime data utilization.

An Evaluation of ReSonAte: A Runtime Risk Assessment Framework for Autonomous Systems

The paper presents ReSonAte, a dynamic risk estimation framework specifically designed for autonomous systems operating within Cyber-Physical Systems (CPS). ReSonAte stands out by addressing the inadequacies of traditional design-time risk assessment techniques, which are static and inflexible when faced with the dynamic and uncertain environments typical of autonomous CPS. This framework incorporates runtime data to continuously update risk estimates, providing a more responsive and adaptable approach to risk assessment crucial for systems that evolve over time.

Key Contributions and Methodology

ReSonAte introduces an extension to the Bow-Tie Diagram (BTD) formalism, adding attributes that model conditional relationships between the system's state and its environment. Such an innovation facilitates the estimation of risk by considering the likelihood of unsafe conditions or system failures derived from safety requirements and runtime data. The paper details a process for scenario modeling, utilizing prior distributions of scenes and threats to generate the necessary data for estimating these conditional relationships.

The evaluation of ReSonAte is demonstrated through two autonomous system simulations: an autonomous ground vehicle (AGV) in the CARLA simulator and an Unmanned Underwater Vehicle (UUV). The AGV example utilizes a perception Learning Enabled Component (LEC) and a custom Scenario Description Language (SDL) to generate varied scenes, allowing the system to adapt to changing environmental and system states. Comprehensive trials with over 600 executions revealed a significant correlation between the dynamic risk estimates by ReSonAte and actual vehicular collisions. Moreover, runtime computation for dynamic risk estimation was efficient, requiring only 0.3 milliseconds on average, underscoring the framework's practicality for real-time applications.

Results and Implications

The results indicate that ReSonAte can dynamically assess collision risks with strong correlation to observed outcomes, thus validating its effectiveness for real-time adaptive risk management within such autonomous systems. The framework extends beyond static risk assessment methods by incorporating environmental and system state changes, which are critical for autonomous systems navigating unpredictable and multifaceted environments. The paper also highlights potential enhancements in risk management processes for CPS, as ReSonAte allows for continuous improvement through runtime data utilization, paving the way for more sophisticated decision-making strategies.

Future Scope

While ReSonAte exhibits promising results, the paper outlines several avenues for further development. These include integrating dynamic assessment of event severity, incorporating state uncertainty for risk confidence bounds, and exploring forecast-based risk estimation. Furthermore, leveraging the risk insights for high-level decisions such as controller adaptations or contingency strategies represents an exciting potential for advancing autonomous systems' operational reliability.

The gradual shift towards more dynamic and contextually aware system assurance methods, epitomized by frameworks like ReSonAte, suggests a future where autonomous systems are not only safer but also more adaptable and intelligent. This paper makes significant strides in that direction, offering a robust foundation for ongoing research and development in runtime risk management for autonomous CPS.