Effectiveness Assessment of Cyber-Physical Systems (1901.06343v4)

Abstract: By achieving their purposes through interactions with the physical world, Cyber-Physical Systems (CPS) pose new challenges in terms of dependability. Indeed, the evolution of the physical systems they control with transducers can be affected by surrounding physical processes over which they have no control and which may potentially hamper the achievement of their purposes. While it is illusory to hope for a comprehensive model of the physical environment at design time to anticipate and remove faults that may occur once these systems are deployed, it becomes necessary to evaluate their degree of effectiveness in vivo. In this paper, the degree of effectiveness is formally defined and generalized in the context of the measure theory. The measure is developed in the context of the Transferable Belief Model (TBM), an elaboration on the Dempster-Shafer Theory (DST) of evidence so as to handle epistemic and aleatory uncertainties respectively pertaining the users' expectations and the natural variability of the physical environment. The TBM is used in conjunction with the Input/Output Hidden Markov Modeling framework (we denote by Ev-IOHMM) to specify the expected evolution of the physical system controlled by the CPS and the tolerances towards uncertainties. The measure of effectiveness is then obtained from the forward algorithm, leveraging the conflict entailed by the successive combinations of the beliefs obtained from observations of the physical system and the beliefs corresponding to its expected evolution. The proposed approach is applied to autonomous vehicles and show how the degree of effectiveness can be used for bench-marking their controller relative to the highway code speed limitations and passengers' well-being constraints, both modeled through an Ev-IOHMM.