A Framework for the Systematic Assessment of Anomaly Detectors in Time-Sensitive Automotive Networks (2405.01324v1)
Abstract: Connected cars are susceptible to cyberattacks. Security and safety of future vehicles highly depend on a holistic protection of automotive components, of which the time-sensitive backbone network takes a significant role. These onboard Time-Sensitive Networks (TSNs) require monitoring for safety and -- as versatile platforms to host Network Anomaly Detection Systems (NADSs) -- for security. Still a thorough evaluation of anomaly detection methods in the context of hard real-time operations, automotive protocol stacks, and domain specific attack vectors is missing along with appropriate input datasets. In this paper, we present an assessment framework that allows for reproducible, comparable, and rapid evaluation of detection algorithms. It is based on a simulation toolchain, which contributes configurable topologies, traffic streams, anomalies, attacks, and detectors. We demonstrate the assessment of NADSs in a comprehensive in-vehicular network with its communication flows, on which we model traffic anomalies. We evaluate exemplary detection mechanisms and reveal how the detection performance is influenced by different combinations of TSN traffic flows and anomaly types. Our approach translates to other real-time Ethernet domains, such as industrial facilities, airplanes, and UAVs.
- Institute of Electrical and Electronics Engineers, “IEEE Standard for Local and Metropolitan Area Network–Bridges and Bridged Networks,” IEEE Std 802.1Q-2022 (Revision of IEEE Std 802.1Q-2018), Dec. 2022.
- S. Checkoway et al., “Comprehensive Experimental Analyses of Automotive Attack Surfaces,” in 20th USENIX Security Symposium, vol. 4. Aug. 2011, pp. 77–92.
- P. Waszecki et al., “Automotive electrical and electronic architecture security via distributed in-vehicle traffic monitoring,” IEEE Trans. on Computer-Aided Design of Integrated Circuits and Systems, vol. 36, pp. 1790–1803, Nov. 2017.
- J.-P. Monteuuis et al., “SARA: Security Automotive Risk Analysis Method,” in 4th ACM Workshop on Cyber-Physical System Security, ser. CPSS ’18. 2018, pp. 3–14.
- I. Sharafaldin et al., “Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization,” in 4th Int. Conf. on Information Systems Security and Privacy (ICISSP), INSTICC. SciTePress, Jan. 2018, pp. 108–116.
- S. Brunner et al., “Automotive E/E-Architecture Enhancements by Usage of Ethernet TSN,” in 13th Workshop on Intelligent Solutions in Embedded Systems (WISES). Jun. 2017, pp. 9–13.
- P. Heise et al., “TSimNet: An Industrial Time Sensitive Networking Simulation Framework Based on OMNeT++,” in 8th IFIP Int. Conf. on New Technologies, Mobility and Sec. (NTMS), Nov. 2016, pp. 1–5.
- J. Jiang et al., “A Time-sensitive Networking (TSN) Simulation Model Based on OMNET++,” in IEEE Int. Conf. on Mechatronics and Automation (ICMA), Aug. 2018, pp. 643–648.
- J. Falk et al., “NeSTiNg: Simulating IEEE Time-sensitive Networking (TSN) in OMNeT++,” in Int. Conf. on Networked Systems (NetSys), Mar. 2019, pp. 1–8.
- P. Meyer et al., “Simulation of Mixed Critical In-vehicular Networks,” in Recent Advances in Network Sim.. Springer, 2019, pp. 317–345.
- M. Rumez et al., “An Overview of Automotive Service-Oriented Architectures and Implications for Security Countermeasures,” IEEE Access, vol. 8, pp. 221 852–221 870, 2020.
- C. Miller and C. Valasek, “Remote Exploitation of an Unaltered Passenger Vehicle,” Black Hat USA, vol. 2015, p. 91, 2015.
- M. Dibaei et al., “Attacks and defences on intelligent connected vehicles: a survey,” Digital Comm. and Networks, vol. 6, pp. 399–421, Nov. 2020.
- I. Pekaric et al., “A taxonomy of attack mechanisms in the automotive domain,” Comp. Standards & Interfaces, vol. 78, p. 103539, Oct. 2021.
- United Nations Economic Commission for Europe, “Draft Recommendation on Cyber Security of the Task Force on Cyber Security and Over-the-air issues of UNECE WP.29 GRVA,” UNECE, Std. WP.29, 2018.
- Int. Organization for Standardization, “Road vehicles – Cybersecurity engineering,” ISO, Geneva, CH, Standard ISO/SAE DIS 21434, 2020.
- P. Meyer et al., “Demo: A Security Infrastructure for Vehicular Information Using SDN, Intrusion Detection, and a Defense Center in the Cloud,” in 2020 IEEE Vehicular Networking Conf. (VNC) . Dec. 2020.
- F. Sommer et al., “Combining Cyber Security Intelligence to Refine Automotive Cyber Threats,” ACM Trans. Priv. Secur., Feb 2024.
- M. H. Bhuyan et al., “Network Anomaly Detection: Methods, Systems and Tools,” IEEE Communications Surveys & Tutorials, vol. 16, pp. 303–336, Jan. 2014.
- G. K. Rajbahadur et al., “A Survey of Anomaly Detection for Connected Vehicle Cybersecurity and Safety,” in IEEE Intelligent Vehicles Symposium (IV). Jun. 2018.
- M. L. Han et al., “TOW-IDS: Intrusion Detection System based on Three Overlapped Wavelets for Automotive Ethernet,” IEEE Trans. on Information Forensics and Security, pp. 1–1, 2022.
- S. Stachowski et al., “An Assessment Method for Automotive Intrusion Detection System Performance,” University of Michigan. Transportation Research Institute, Tech. Rep. DOT HS 812 708, Apr. 2019.
- A. Thakkar and R. Lohiya, “A Review of the Advancement in Intrusion Detection Datasets,” Procedia Comp. Science, vol. 167, pp. 636–645, 2020, Int. Conf. on Computational Intelligence and Data Science.
- S. Lee et al., “A Comprehensive Analysis of Datasets for Automotive Intrusion Detection Systems” Comp., Materials & Continua, vol. 76,2023.
- N. Rajasinghe et al., “INSECS-DCS: A Highly Customizable Network Intrusion Dataset Creation Framework,” in 2018 IEEE Canadian Conf. on Electrical & Computer Engineering (CCECE). IEEE, May 2018.
- J. Lauinger et al., “Attack Data Generation Framework for Autonomous Vehicle Sensors,” in Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, Mar 2022.
- A. Ibraheem et al., “Neural Network based Partial Tomography for In-Vehicle Network Monitoring,” in IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, Jun 2021.
- J. McHugh, “Testing Intrusion Detection Systems: A Critique of the 1998 and 1999 DARPA Intrusion Detection System Evaluations as Performed by Lincoln Laboratory,” ACM Trans. Inf. Syst. Secur., vol. 3, p. 262–294, Nov 2000.
- A. Finkenzeller et al., “Simutack - An Attack Simulation Framework for Connected and Autonomous Vehicles,” in 97th IEEE Vehicular Technology Conference (VTC2023-Spring), Jun 2023, pp. 1–7.