Explainable Machine Learning: The Importance of a System-Centric Perspective

Abstract: The landscape in the context of several signal processing applications and even education appears to be significantly affected by the emergence of ML and in particular deep learning (DL).The main reason for this is the ability of DL to model complex and unknown relationships between signals and the tasks of interest. Particularly, supervised DL algorithms have been fairly successful at recognizing perceptually or semantically useful signal information in different applications. In all of these, the training process uses labeled data to learn a mapping function (typically implicitly) from signals to the desired information (class label or target label). The trained DL model is then expected to correctly recognize/classify relevant information in a given test signal. A DL based framework is therefore, in general, very appealing since the features and characteristics of the required mapping are learned almost exclusively from the data without resorting to explicit model/system development. The focus on implicit modeling however also raises the issue of lack of explainability/interpretability of the resultant DL based mapping or the black box problem. As a result, explainable ML/DL is an active research area where the primary goal is to elaborate how the ML/DL model arrived at a prediction. We however note that despite the efforts, the commentary on black box problem appears to lack a technical discussion from the view point of: a) its origin and underlying reasons, and b) its practical implications on the design and deployment of ML/DL systems. Accordingly, a reasonable question that can be raised is as follows. Can the traditional system-centric approach (which places emphasis on explicit system modeling) provide useful insights into the nature of black box problem, and help develop more transparent ML/DL systems?