Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RainSD: Rain Style Diversification Module for Image Synthesis Enhancement using Feature-Level Style Distribution (2401.00460v1)

Published 31 Dec 2023 in cs.CV

Abstract: Autonomous driving technology nowadays targets to level 4 or beyond, but the researchers are faced with some limitations for developing reliable driving algorithms in diverse challenges. To promote the autonomous vehicles to spread widely, it is important to address safety issues on this technology. Among various safety concerns, the sensor blockage problem by severe weather conditions can be one of the most frequent threats for multi-task learning based perception algorithms during autonomous driving. To handle this problem, the importance of the generation of proper datasets is becoming more significant. In this paper, a synthetic road dataset with sensor blockage generated from real road dataset BDD100K is suggested in the format of BDD100K annotation. Rain streaks for each frame were made by an experimentally established equation and translated utilizing the image-to-image translation network based on style transfer. Using this dataset, the degradation of the diverse multi-task networks for autonomous driving, such as lane detection, driving area segmentation, and traffic object detection, has been thoroughly evaluated and analyzed. The tendency of the performance degradation of deep neural network-based perception systems for autonomous vehicle has been analyzed in depth. Finally, we discuss the limitation and the future directions of the deep neural network-based perception algorithms and autonomous driving dataset generation based on image-to-image translation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Did we test all scenarios for automated and autonomous driving systems?, in: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), IEEE, 2019, pp. 2950–2955.
  2. S. Ionita, Autonomous vehicles: from paradigms to technology, in: IOP Conference Series: Materials Science and Engineering, volume 252, IOP Publishing, 2017, p. 012098.
  3. C. Gkartzonikas, K. Gkritza, What have we learned? a review of stated preference and choice studies on autonomous vehicles, Transportation Research Part C: Emerging Technologies 98 (2019) 323–337.
  4. Sensor and sensor fusion technology in autonomous vehicles: A review, Sensors 21 (2021) 2140.
  5. The determinants behind the acceptance of autonomous vehicles: A systematic review, Sustainability 12 (2020) 1719.
  6. K. Othman, Public acceptance and perception of autonomous vehicles: a comprehensive review, AI and Ethics 1 (2021) 355–387.
  7. A review on safety failures, security attacks, and available countermeasures for autonomous vehicles, Ad Hoc Networks 90 (2019) 101823.
  8. Effect of adherent rain on vision-based object detection algorithms, SAE International Journal of Advances and Current Practices in Mobility 2 (2020) 3051–3059.
  9. M. Hnewa, H. Radha, Object detection under rainy conditions for autonomous vehicles: A review of state-of-the-art and emerging techniques, IEEE Signal Processing Magazine 38 (2020) 53–67.
  10. Analysis of lidar and camera data in real-world weather conditions for autonomous vehicle operations, SAE International Journal of Advances and Current Practices in Mobility 2 (2020) 2428–2434.
  11. Automated driving recognition technologies for adverse weather conditions, IATSS research 43 (2019) 253–262.
  12. E. Zio, The future of risk assessment, Reliability Engineering & System Safety 177 (2018) 176–190.
  13. Performance test of autonomous vehicle lidar sensors under different weather conditions, Transportation research record 2674 (2020) 319–329.
  14. Realistic lidar with noise model for real-time testing of automated vehicles in a virtual environment, IEEE Sensors Journal 21 (2021) 9919–9926.
  15. Cycada: Cycle-consistent adversarial domain adaptation, in: International conference on machine learning, Pmlr, 2018, pp. 1989–1998.
  16. Unsupervised image-to-image translation networks, Advances in neural information processing systems 30 (2017).
  17. Forkgan: Seeing into the rainy night, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, Springer, 2020, pp. 155–170.
  18. Toward multimodal image-to-image translation, Advances in neural information processing systems 30 (2017).
  19. Image-to-image translation for autonomous driving from coarsely-aligned image pairs, in: 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2023, pp. 7756–7762.
  20. Night-to-day: Online image-to-image translation for object detection within autonomous driving by night, IEEE Transactions on Intelligent Vehicles 6 (2020) 480–489.
  21. Cross-domain car detection using unsupervised image-to-image translation: From day to night, in: 2019 International Joint Conference on Neural Networks (IJCNN), IEEE, 2019, pp. 1–8.
  22. Gan-based day-to-night image style transfer for nighttime vehicle detection, IEEE Transactions on Intelligent Transportation Systems 22 (2020) 951–963.
  23. Raidar: A rich annotated image dataset of rainy street scenes, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 2951–2961.
  24. Memory-guided unsupervised image-to-image translation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 6558–6567.
  25. Bdd100k: A diverse driving dataset for heterogeneous multitask learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2636–2645.
  26. Tsit: A simple and versatile framework for image-to-image translation, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, Springer, 2020, pp. 206–222.
  27. The cityscapes dataset for semantic urban scene understanding, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3213–3223.
  28. Vision meets robotics: The kitti dataset, The International Journal of Robotics Research 32 (2013) 1231–1237.
  29. nuscenes: A multimodal dataset for autonomous driving, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11621–11631.
  30. The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3234–3243.
  31. M. Wrenninge, J. Unger, Synscapes: A photorealistic synthetic dataset for street scene parsing, arXiv preprint arXiv:1810.08705 (2018).
  32. Carla: An open urban driving simulator, in: Conference on robot learning, PMLR, 2017, pp. 1–16.
  33. Lgsvl simulator: A high fidelity simulator for autonomous driving, in: 2020 IEEE 23rd International conference on intelligent transportation systems (ITSC), IEEE, 2020, pp. 1–6.
  34. Airsim: High-fidelity visual and physical simulation for autonomous vehicles, in: Field and Service Robotics: Results of the 11th International Conference, Springer, 2018, pp. 621–635.
  35. Carla simulator-based evaluation framework development of lane detection accuracy performance under sensor blockage caused by heavy rain for autonomous vehicle, IEEE Robotics and Automation Letters 7 (2022) 9977–9984.
  36. Improving semantic segmentation via video propagation and label relaxation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 8856–8865.
  37. Hierarchical multi-scale attention for semantic segmentation, arXiv preprint arXiv:2005.10821 (2020).
  38. Pointrend: Image segmentation as rendering, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9799–9808.
  39. Tensormask: A foundation for dense object segmentation, in: Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 2061–2069.
  40. Image-to-image translation: Methods and applications, IEEE Transactions on Multimedia 24 (2021) 3859–3881.
  41. Diverse image-to-image translation via disentangled representations, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 35–51.
  42. Effects of sim2real image translation via dclgan on lane keeping assist system in carla simulator, IEEE Access 11 (2023) 33915–33927.
  43. Domain bridge for unpaired image-to-image translation and unsupervised domain adaptation, in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2020, pp. 2990–2998.
  44. Image-to-image translation with conditional adversarial networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1125–1134.
  45. Unpaired image-to-image translation using cycle-consistent adversarial networks, in: Proceedings of the IEEE international conference on computer vision, 2017, pp. 2223–2232.
  46. Multimodal unsupervised image-to-image translation, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 172–189.
  47. Drit++: Diverse image-to-image translation via disentangled representations, International Journal of Computer Vision 128 (2020) 2402–2417.
  48. Multi-task learning for dangerous object detection in autonomous driving, Information Sciences 432 (2018) 559–571.
  49. Y. Zhang, Q. Yang, A survey on multi-task learning, IEEE Transactions on Knowledge and Data Engineering 34 (2021) 5586–5609.
  50. End-to-end multi-task learning with attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 1871–1880.
  51. Yolop: You only look once for panoptic driving perception, Machine Intelligence Research 19 (2022) 550–562.
  52. Hybridnets: End-to-end perception network, arXiv preprint arXiv:2203.09035 (2022).
  53. Perceptual losses for real-time style transfer and super-resolution, in: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, Springer, 2016, pp. 694–711.
  54. High-resolution image synthesis and semantic manipulation with conditional gans, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8798–8807.
  55. Pytorch: An imperative style, high-performance deep learning library, Advances in neural information processing systems 32 (2019).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Hyeonjae Jeon (1 paper)
  2. Junghyun Seo (2 papers)
  3. Taesoo Kim (30 papers)
  4. Sungho Son (2 papers)
  5. Jungki Lee (1 paper)
  6. Gyeungho Choi (6 papers)
  7. Yongseob Lim (7 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets