Learning Signal Temporal Logic through Neural Network for Interpretable Classification (2210.01910v2)
Abstract: Machine learning techniques using neural networks have achieved promising success for time-series data classification. However, the models that they produce are challenging to verify and interpret. In this paper, we propose an explainable neural-symbolic framework for the classification of time-series behaviors. In particular, we use an expressive formal language, namely Signal Temporal Logic (STL), to constrain the search of the computation graph for a neural network. We design a novel time function and sparse softmax function to improve the soundness and precision of the neural-STL framework. As a result, we can efficiently learn a compact STL formula for the classification of time-series data through off-the-shelf gradient-based tools. We demonstrate the computational efficiency, compactness, and interpretability of the proposed method through driving scenarios and naval surveillance case studies, compared with state-of-the-art baselines.
- U. R. Acharya, S. L. Oh, Y. Hagiwara, J. H. Tan, M. Adam, A. Gertych, and R. San Tan, “A deep convolutional neural network model to classify heartbeats,” Computers in biology and medicine, vol. 89, pp. 389–396, 2017.
- D. Tang, B. Qin, and T. Liu, “Document modeling with gated recurrent neural network for sentiment classification,” in Proceedings of the 2015 conference on empirical methods in natural language processing, 2015, pp. 1422–1432.
- O. Maler and D. Nickovic, “Monitoring temporal properties of continuous signals,” in Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems. Springer, 2004, pp. 152–166.
- V. Raman, A. Donzé, M. Maasoumy, R. M. Murray, A. Sangiovanni-Vincentelli, and S. A. Seshia, “Model predictive control with signal temporal logic specifications,” in 53rd IEEE Conference on Decision and Control. IEEE, 2014, pp. 81–87.
- M. Ma, J. Gao, L. Feng, and J. Stankovic, “Stlnet: Signal temporal logic enforced multivariate recurrent neural networks,” Advances in Neural Information Processing Systems, vol. 33, pp. 14 604–14 614, 2020.
- X. Jin, A. Donzé, J. V. Deshmukh, and S. A. Seshia, “Mining requirements from closed-loop control models,” in Proceedings of the 16th international conference on Hybrid systems: computation and control, 2013, pp. 43–52.
- A. Bakhirkin, T. Ferrère, and O. Maler, “Efficient parametric identification for stl,” in Proceedings of the 21st International Conference on Hybrid Systems: Computation and Control (part of CPS Week), 2018, pp. 177–186.
- B. Hoxha, A. Dokhanchi, and G. Fainekos, “Mining parametric temporal logic properties in model-based design for cyber-physical systems,” International Journal on Software Tools for Technology Transfer, vol. 20, no. 1, pp. 79–93, 2018.
- Z. Kong, A. Jones, and C. Belta, “Temporal logics for learning and detection of anomalous behavior,” IEEE Transactions on Automatic Control, vol. 62, no. 3, pp. 1210–1222, 2016.
- G. Bombara, C.-I. Vasile, F. Penedo, H. Yasuoka, and C. Belta, “A decision tree approach to data classification using signal temporal logic,” in Proceedings of the 19th International Conference on Hybrid Systems: Computation and Control, 2016, pp. 1–10.
- S. Mohammadinejad, J. V. Deshmukh, A. G. Puranic, M. Vazquez-Chanlatte, and A. Donzé, “Interpretable classification of time-series data using efficient enumerative techniques,” in Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control, 2020, pp. 1–10.
- E. Aasi, C. I. Vasile, M. Bahreinian, and C. Belta, “Classification of time-series data using boosted decision trees,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 1263–1268.
- J.-R. Gaglione, D. Neider, R. Roy, U. Topcu, and Z. Xu, “Maxsat-based temporal logic inference from noisy data,” Innovations in Systems and Software Engineering, vol. 18, no. 3, pp. 427–442, 2022.
- T. Rocktäschel and S. Riedel, “End-to-end differentiable proving,” Advances in neural information processing systems, vol. 30, 2017.
- F. Yang, Z. Yang, and W. W. Cohen, “Differentiable learning of logical rules for knowledge base reasoning,” Advances in neural information processing systems, vol. 30, 2017.
- K. Leung, N. Aréchiga, and M. Pavone, “Back-propagation through signal temporal logic specifications: Infusing logical structure into gradient-based methods,” in International Workshop on the Algorithmic Foundations of Robotics. Springer, 2020, pp. 432–449.
- A. Ketenci and E. A. Gol, “Learning parameters of ptstl formulas with backpropagation,” in 2020 28th Signal Processing and Communications Applications Conference (SIU). IEEE, 2020, pp. 1–4.
- R. Yan and A. Julius, “Neural network for weighted signal temporal logic,” arXiv preprint arXiv:2104.05435, 2021.
- N. Baharisangari, K. Hirota, R. Yan, A. Julius, and Z. Xu, “Weighted graph-based signal temporal logic inference using neural networks,” IEEE Control Systems Letters, vol. 6, pp. 2096–2101, 2021.
- G. Chen, Y. Lu, R. Su, and Z. Kong, “Interpretable fault diagnosis of rolling element bearings with temporal logic neural network,” arXiv preprint arXiv:2204.07579, 2022.
- N. Mehdipour, C.-I. Vasile, and C. Belta, “Arithmetic-geometric mean robustness for control from signal temporal logic specifications,” in 2019 American Control Conference (ACC). IEEE, 2019, pp. 1690–1695.
- ——, “Specifying user preferences using weighted signal temporal logic,” IEEE Control Systems Letters, vol. 5, no. 6, pp. 2006–2011, 2021.
- S. Srinivas, A. Subramanya, and R. Venkatesh Babu, “Training sparse neural networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 138–145.
- Y. Bengio, N. Léonard, and A. Courville, “Estimating or propagating gradients through stochastic neurons for conditional computation,” arXiv preprint arXiv:1308.3432, 2013.