Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Runtime Stealthy Perception Attacks against DNN-based Adaptive Cruise Control Systems (2307.08939v4)

Published 18 Jul 2023 in cs.CR, cs.CV, and cs.LG

Abstract: Adaptive Cruise Control (ACC) is a widely used driver assistance technology for maintaining the desired speed and safe distance to the leading vehicle. This paper evaluates the security of the deep neural network (DNN) based ACC systems under runtime stealthy perception attacks that strategically inject perturbations into camera data to cause forward collisions. We present a context-aware strategy for the selection of the most critical times for triggering the attacks and a novel optimization-based method for the adaptive generation of image perturbations at runtime. We evaluate the effectiveness of the proposed attack using an actual vehicle, a publicly available driving dataset, and a realistic simulation platform with the control software from a production ACC system, a physical-world driving simulator, and interventions by the human driver and safety features such as Advanced Emergency Braking System (AEBS). Experimental results show that the proposed attack achieves 142.9 times higher success rate in causing hazards and 82.6% higher evasion rate than baselines, while being stealthy and robust to real-world factors and dynamic changes in the environment. This study highlights the role of human drivers and basic safety mechanisms in preventing attacks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (94)
  1. “Adas cameras: How they work and why they need calibration,” https://caradas.com/adas-cameras/.
  2. “ADAS user study,” https://drive.google.com/file/d/1GtMQpmgIzu4ZcjRbYKQfdE1q8WEEYPue/view?usp=sharing.
  3. “comma connect,” https://www.comma.ai/connect.
  4. “Cybersecurity risks for hi-tech autonomous and electric vehicles industry,” https://www.linkedin.com/pulse/cybersecurity-risks-hi-tech-autonomous-electric-vehicles-samrat-seal/.
  5. “GRVA-12-50r1e.pdf,” https://unece.org/sites/default/files/2022-01/GRVA-12-50r1e.pdf.
  6. “Installing a fork of openpilot with workbench,” https://medium.com/@jfrux/installing-a-fork-of-openpilot-with-workbench-de35e9388021.
  7. “Panda,” https://github.com/commaai/panda.
  8. “Qualtrics,” https://www.qualtrics.com/.
  9. “Supported Cars by OpenPilot,” https://github.com/commaai/openpilot/blob/master/docs/CARS.md.
  10. “Tesla autopilot,” https://www.tesla.com/autopilot.
  11. “Safety Architecture,” {https://blog.comma.ai/how-to-write-a-car-port-for-openpilot/#background--safety-architecture}, 2018.
  12. “Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles,” SAE international, vol. 4970, no. 724, pp. 1–5, 2018.
  13. “Openpilot: An overview and the port to the honda clarity: Hardware,” {https://wirelessnet2.medium.com/openpilot-an-overview-and-the-port-to-the-honda-clarity-16341d53c9aa}, 2020.
  14. “UN Regulation No 152 – Uniform provisions concerning the approval of motor vehicles with regard to the Advanced Emergency Braking System (AEBS) for M1 and N1 vehicles [2020/1597],” http://data.europa.eu/eli/reg/2020/1597/oj, pp. 66–89, 2020.
  15. “Adaptive Cruise Control (ACC) Operating Characteristics and User Interface: Standard J2399,” Society of Automotive Engineers, 2021.
  16. “BMW 3 Series Dimensions,” {https://www.carsguide.com.au/bmw/3-series/car-dimensions/2021}, 2021.
  17. “Openpilot ssh key security bypass,” https://www.redpacketsecurity.com/openpilot-ssh-key-security-bypass/, 2021.
  18. “SAE Levels of Driving Automation™ Refined for Clarity and International Audience,” https://www.sae.org/blog/sae-j3016-update, 2021.
  19. “Number of autonomous vehicles globally in 2022,” 2022. [Online]. Available: https://www.statista.com/statistics/1230664/projected-number-autonomous-cars-worldwide/
  20. T. Alsuwian, R. B. Saeed, and A. A. Amin, “Autonomous Vehicle with Emergency Braking Algorithm Based on Multi-Sensor Fusion and Super Twisting Speed Controller,” Applied Sciences, vol. 12, no. 17, p. 8458, Aug. 2022.
  21. Baidu, “Apollo,” https://developer.apollo.auto/.
  22. G. Bishop, G. Welch et al., “An introduction to the kalman filter,” Proc of SIGGRAPH, Course, vol. 8, no. 27599-23175, p. 41, 2001.
  23. A. Chahe, C. Wang et al., “Dynamic adversarial attacks on autonomous driving systems,” arXiv preprint arXiv:2312.06701, 2023.
  24. L. Chen, T. Tang et al., “Level 2 autonomous driving on a single device: Diving into the devils of openpilot,” arXiv:2206.08176, 2022.
  25. Z. Chen, P. Dash, and K. Pattabiraman, “Jujutsu: A Two-stage Defense against Adversarial Patch Attacks on Deep Neural Networks,” in Proceedings of the ACM Asia Conference on Computer and Communications Security.   ACM, Jul. 2023, pp. 689–703.
  26. H. Choi, S. Kate et al., “Software-based realtime recovery from sensor attacks on robotic vehicles,” in 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID), 2020, pp. 349–364.
  27. Comma.ai, “Openpilot,” https://comma.ai/openpilot.
  28. Consumer Reports, “CR Active Driving Assistance Systems: Test Results & Design Recommendations,” https://data.consumerreports.org/reports/cr-active-driving-assistance-systems/.
  29. F. Croce and M. Hein, “Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,” in International conference on machine learning.   PMLR, 2020, pp. 2206–2216.
  30. A. Dosovitskiy, G. Ros et al., “CARLA: An open urban driving simulator,” in Proceedings of the 1st Annual Conference on Robot Learning, 2017, pp. 1–16.
  31. G. K. Dziugaite, Z. Ghahramani, and D. M. Roy, “A study of the effect of jpg compression on adversarial images,” arXiv:1608.00853, 2016.
  32. M. H. Eiza and Q. Ni, “Driving with sharks: Rethinking connected vehicles with vehicle cybersecurity,” IEEE Vehicular Technology Magazine, vol. 12, no. 2, pp. 45–51, 2017.
  33. A. A. Elkhail, R. U. D. Refat et al., “Vehicle security: A survey of security issues and vulnerabilities, malware attacks and defenses,” IEEE Access, vol. 9, pp. 162 401–162 437, 2021.
  34. K. Eykholt, I. Evtimov et al., “Robust physical-world attacks on deep learning visual classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1625–1634.
  35. I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv:1412.6572, 2014.
  36. A. Greenberg, “Hackers remotely kill a jeep on the highway—with me in it,” Wired, 2015.
  37. Y. Guo, T. Sato et al., “Adversarial attacks on adaptive cruise control systems,” in Proceedings of Cyber-Physical Systems and IoT Week 2023, 2023, pp. 49–54.
  38. R. S. Hallyburton, Y. Liu et al., “Security analysis of camera-lidar fusion against black-box attacks on autonomous vehicles,” in 31st USENIX Security Symposium (USENIX Security 22), 2022, pp. 1903–1920.
  39. S. Hoory, T. Shapira et al., “Dynamic adversarial patch for evading object detection models,” arXiv:2010.13070, 2020.
  40. J. Hunt and J. Hunt, “Monkey patching and attribute lookup,” A Beginners Guide to Python 3 Programming, pp. 325–336, 2019.
  41. S. Jha, S. Banerjee et al., “Ml-based fault injection for autonomous vehicles: A case for bayesian fault injection,” in 2019 49th annual IEEE/IFIP international conference on dependable systems and networks (DSN), 2019, pp. 112–124.
  42. S. Jha, S. Cui et al., “Ml-driven malware that targets av safety,” in 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2020, pp. 113–124.
  43. Y. Jia, Y. Lu et al., “Fooling detection alone is not enough: First adversarial attack against multiple object tracking,” arXiv:1905.11026, 2019.
  44. K. Kim, J. S. Kim et al., “Cybersecurity for autonomous vehicles: Review of attacks and defense,” Computers &\&& Security, vol. 103, p. 102150, 2021.
  45. R. Komissarov and A. Wool, “Spoofing attacks against vehicular fmcw radar,” in Proceedings of the 5th Workshop on Attacks and Solutions in Hardware Security, 2021, pp. 91–97.
  46. K. Koscher, A. Czeskis et al., “Experimental security analysis of a modern automobile,” in 2010 IEEE symposium on security and privacy, 2010, pp. 447–462.
  47. A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” in Artificial intelligence safety and security.   Chapman and Hall/CRC, 2018, pp. 99–112.
  48. S. Lagraa, M. Cailac et al., “Real-time attack detection on robot cameras: A self-driving car application,” in 2019 Third IEEE International Conference on Robotic Computing (IRC).   IEEE, 2019, pp. 102–109.
  49. M. Lee and Z. Kolter, “On physical adversarial patches for object detection,” arXiv:1906.11897, 2019.
  50. N. Leveson and J. Thomas, “An stpa primer,” Cambridge, MA, 2013.
  51. J. Li, F. Schmidt, and Z. Kolter, “Adversarial camera stickers: A physical camera-based attack on deep learning systems,” in International Conference on Machine Learning, 2019, pp. 3896–3904.
  52. J. Liu, A. Levine et al., “Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).   IEEE, Jun. 2022, pp. 14 953–14 962.
  53. X. Liu, H. Yang et al., “Dpatch: An adversarial patch attack on object detectors,” arXiv:1806.02299, 2018.
  54. G. Lovisotto, H. Turner et al., “SLAP: Improving physical adversarial examples with short-lived adversarial perturbations,” in 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 1865–1882.
  55. C. Ma, N. Wang et al., “WIP: Towards the Practicality of the Adversarial Attack on Object Tracking in Autonomous Driving,” in Inaugural International Symposium on Vehicle Security & Privacy, 2023.
  56. ——, “Wip: Towards the practicality of the adversarial attack on object tracking in autonomous driving,” in ISOC Symposium on Vehicle Security and Privacy (VehicleSec), 2023.
  57. Y. Ma, J. A. Sharp et al., “Sequential attacks on kalman filter-based forward collision warning systems,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 10, 2021, pp. 8865–8873.
  58. A. Madry, A. Makelov et al., “Towards deep learning models resistant to adversarial attacks,” arXiv:1706.06083, 2017.
  59. Y. Man, R. Muller et al., “That person moves like a car: Misclassification attack detection for autonomous systems using spatiotemporal consistency,” in 32nd USENIX Security Symposium (USENIX Security 23), 2023, pp. 6929–6946.
  60. C. Miller and C. Valasek, “Remote exploitation of an unaltered passenger vehicle,” Black Hat USA, vol. 2015, no. S 91, pp. 1–91, 2015.
  61. R. Mocnik, D. S. Fowler, and C. Maple, “Vehicular Over-the-Air Software Upgrade Threat Modelling,” in Cenex-LCV and Cenex-CAM 2023. [Online]. Available: https://wrap.warwick.ac.uk/179188/1/WRAP-vehicular-over-the-air-software-upgrade-threat-modelling-2023.pdf
  62. S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: a simple and accurate method to fool deep neural networks,” in IEEE conference on computer vision and pattern recognition, 2016, pp. 2574–2582.
  63. M. Moradi, B. J. Oakes et al., “Exploring fault parameter space using reinforcement learning-based fault injection,” in 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), 2020, pp. 102–109.
  64. W. G. Najm, J. D. Smith et al., “Pre-crash scenario typology for crash avoidance research,” Tech. Rep. DOT-VNTSC-NHTSA-06-02, Apr. 2007.
  65. S. Nie, L. Liu, and Y. Du, “Free-fall: Hacking tesla from wireless to can bus,” Briefing, Black Hat USA, vol. 25, pp. 1–16, 2017.
  66. N. Papernot, P. McDaniel et al., “Towards the science of security and privacy in machine learning,” arXiv:1611.03814, 2016.
  67. R. Ravindran, M. J. Santora, and M. M. Jamali, “Multi-object detection and tracking, based on dnn, for autonomous vehicles: A review,” IEEE Sensors Journal, vol. 21, no. 5, pp. 5668–5677, 2020.
  68. J. Redmon, S. Divvala et al., “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779–788.
  69. D. Rezvani. Hacking automotive ethernet cameras. [Online]. Available: https://argus-sec.com/hacking-automotive-ethernet-cameras/
  70. A. H. M. Rubaiyat, Y. Qin, and H. Alemzadeh, “Experimental resilience assessment of an open-source driving agent,” in IEEE Pacific rim international symposium on dependable computing, 2018, pp. 54–63.
  71. T. Sato, S. H. V. Bhupathiraju et al., “WIP: Infrared Laser Reflection Attack Against Traffic Sign Recognition Systems,” in Proceedings Inaugural International Symposium on Vehicle Security & Privacy, 2023.
  72. T. Sato, Y. Hayakawa et al., “WIP: Practical Removal Attacks on LiDAR-based Object Detection in Autonomous Driving,” in Inaugural International Symposium on Vehicle Security & Privacy, 2023.
  73. T. Sato, J. Shen et al., “Dirty road can attack: Security of deep learning based automated lane centering under physical-world attack,” in 30th USENIX Security Symposium, 2021, pp. 3309–3326.
  74. H. Schafer, E. Santana et al., “A commute in data: The comma2k19 dataset,” arXiv:1812.05752, 2018.
  75. R. Schram, A. Williams, and M. van Ratingen, “Implementation of autonomous emergency braking (aeb), the next step in euro ncap’s safety assessment,” ESV, Seoul, 2013.
  76. E. Schubert, J. Sander et al., “Dbscan revisited, revisited: why and how you should (still) use dbscan,” ACM Transactions on Database Systems (TODS), vol. 42, no. 3, pp. 1–21, 2017.
  77. S. Sharma, J. A. Ansari et al., “Beyond pixels: Leveraging geometry and shape cues for online multi-object tracking,” in 2018 IEEE International Conference on Robotics and Automation, 2018, pp. 3508–3515.
  78. J. Shen, J. Y. Won et al., “Drift with devil: Security of multi-sensor fusion based localization in high-level autonomous driving under gps spoofing,” in Proceedings of the 29th USENIX Conference on Security Symposium, 2020, pp. 931–948.
  79. E. Shi, “Openpilot: An overview and the port to the honda clarity: Hardware,” https://wirelessnet2.medium.com/openpilot-an-overview-and-the-port-to-the-honda-clarity-16341d53c9aa.
  80. M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in International conference on machine learning, 2017, pp. 3319–3328.
  81. Tencent, “Experimental security research of tesla autopilot,” Tencent Keen Security Lab, 2019.
  82. I. Urazghildiiev, R. Ragnarsson et al., “Vehicle classification based on the radar measurement of height profiles,” IEEE Transactions on intelligent transportation systems, vol. 8, no. 2, pp. 245–253, 2007.
  83. Z. Wang and A. C. Bovik, “A universal image quality index,” IEEE signal processing letters, vol. 9, no. 3, pp. 81–84, 2002.
  84. H. Wen, Q. A. Chen, and Z. Lin, “Plug-n-pwned: Comprehensive vulnerability analysis of OBD-II dongles as a new over-the-air attack surface in automotive iot,” in 29th USENIX security symposium (USENIX Security 20), 2020, pp. 949–965.
  85. B. Weng, M. Zhu, and K. Redmill, “A formal safety characterization of advanced driver assist systems in the car-following regime with scenario-sampling,” IFAC-PapersOnLine, vol. 55 no.24, pp. 266–272, 2022.
  86. Wikipedia, “Norm,” https://en.wikipedia.org/wiki/Norm_(mathematics).
  87. Z. Wu, S.-N. Lim et al., “Making an invisibility cloak: Real world adversarial attacks on object detectors,” in Computer Vision-ECCV, 2020, pp. 1–17.
  88. C. Xiang, A. N. Bhagoji et al., “Patchguard: A provably robust defense against adversarial patches via small receptive fields and masking,” in 30th USENIX Security Symposium, 2021, pp. 2237–2254.
  89. K. Xu, Y. Xiao et al., “PatchZero: Defending against Adversarial Patch Attacks by Detecting and Zeroing the Patch,” in IEEE/CVF Winter Conference on Applications of Computer Vision, Jan. 2023, pp. 4621–4630.
  90. W. Xu, D. Evans, and Y. Qi, “Feature squeezing: Detecting adversarial examples in deep neural networks,” arXiv:1704.01155, 2017.
  91. Y. Zhang and P. Liang, “Defending against whitebox adversarial attacks via randomized discretization,” in The 22nd International Conference on Artificial Intelligence and Statistics, 2019, pp. 684–693.
  92. S. Zhou, C. Liu et al., “Adversarial attacks and defenses in deep learning: From a perspective of cybersecurity,” ACM Computing Surveys, vol. 55, no. 8, pp. 1–39, 2022.
  93. X. Zhou, B. Ahmed et al., “Hybrid knowledge and data driven synthesis of runtime monitors for cyber-physical systems,” IEEE Transactions on Dependable and Secure Computing, 2023.
  94. X. Zhou, A. Schmedding et al., “Strategic safety-critical attacks against an advanced driver assistance system,” in 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2022, pp. 79–87.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now