Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey (2112.08936v2)

Abstract: Automated Driving Systems (ADS) open up a new domain for the automotive industry and offer new possibilities for future transportation with higher efficiency and comfortable experiences. However, autonomous driving under adverse weather conditions has been the problem that keeps autonomous vehicles (AVs) from going to level 4 or higher autonomy for a long time. This paper assesses the influences and challenges that weather brings to ADS sensors in an analytic and statistical way, and surveys the solutions against inclement weather conditions. State-of-the-art techniques on perception enhancement with regard to each kind of weather are thoroughly reported. External auxiliary solutions, weather conditions coverage in currently available datasets, simulators, and experimental facilities with weather chambers are distinctly sorted out. Additionally, potential future ADS sensors candidates and approaches beyond common senses are provided. By looking into all kinds of major weather problems the autonomous driving field is currently facing, and reviewing both hardware and computer science solutions in recent years, this survey points out the main moving trends of adverse weather problems in autonomous driving, i.e., advanced sensor fusions, more sophisticated networks, and V2X & IoT technologies; and also the limitations brought by emerging 1550 nm LiDARs. In general, this work contributes a holistic overview of the obstacles and directions of ADS development in terms of adverse weather driving conditions.

Autonomous Vehicle Perception and Sensing in Adverse Weather: An Academic Overview

The paper "Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey" presents a comprehensive review of the challenges and solutions for autonomous vehicle (AV) perception systems operating in unfavorable weather conditions. The persistent issue that restricts AVs from achieving level 4 or higher autonomy, according to SAE standards, stems primarily from their diminished performance in adverse weather. This comprehensive survey assesses the impacts of weather on Automated Driving Systems (ADS) sensors and evaluates current technological countermeasures.

The survey meticulously categorizes state-of-the-art perception enhancement strategies tailored to various weather conditions, such as rain, snow, fog, and strong light. It also explores the integration of auxiliary solutions, analyses the representation of different weather conditions in current datasets, and reviews experimental setups like weather chambers. Moreover, potential future sensor technologies and unconventional approaches are evaluated, highlighting advanced sensor fusions, sophisticated network models, and the role of V2X and IoT technologies.

Perception and Sensor Fusion Technologies

The paper emphasizes the indispensability of sensor fusion for robust AV operation under adverse weather. LiDAR, radar, cameras, and other sensors each have inherent strengths and vulnerabilities when encountering weather-induced challenges. The complex interplay between sensor types and fusion algorithms forms the basis for mitigating perception degradation due to conditions such as rain or snow.

1. LiDAR: Although reliable in typical conditions, LiDAR's performance can be impaired due to signal attenuation and noise induced by rain or fog. The report examines strategies like multi-echo processing and waveform identification to filter out false detections in weather-afflicted point clouds. It also discusses the limitations facing new LiDAR technologies, such as Fiducial Mark Correlation Wave (FMCW) and 1550 nm LiDARs.
2. Radar: Known for its superior resilience to adverse weather, especially wet conditions, radar also struggles with low spatial resolution, hindering object classification without fusion with other sensors. Innovations in radar's integration into multimodal fusion strategies are highlighted.
3. Cameras: Cameras are crucial yet highly susceptible to adverse weather impacts. Advanced de-raining and de-hazing techniques using deep learning are detailed, showcasing how these restoration models are vital for maintaining image clarity. Enhanced thermal cameras and event-based cameras also feature as key future considerations for camera-based perception systems.
4. Sensor Fusion: Advanced sensor fusion modalities, which efficiently leverage radar, thermal photography, and robust LiDAR integration, are explored. These combinations are necessary to balance weaknesses and strengths, enhancing situational awareness and data redundancy.

Implications and Future Directions

The synthesis of environmental perception advancements pushes the boundary of AV capabilities in challenging weather. The work points out the promise of V2X implementations, where communication networks extend the horizon of vehicular perception capabilities, fostering a more interconnected and intelligent transport infrastructure. Embedded systems such as auxiliary roadside units add another layer of reliability by providing independent verification of sensory inputs.

The paper identifies several underexplored avenues for future research, focusing on the potential of burgeoning technologies like hyperspectral imaging and high dynamic range (HDR) cameras for breakthrough AV perception capabilities. Additionally, importance is placed on enhancing the fiduciary data ecosystems—both in terms of acquisition and processing—to close the quality gap facing current datasets.

Conclusion

Through this detailed survey, Zhang et al. aim to illuminate the multifaceted obstacles faced by AVs under adverse weather and elucidate the pathways researchers and engineers can take to surmount these challenges. This insightful compilation not only provides a trove of technical insights but also serves as a guiding framework for current and future developments in AV perception technology. The exploration of existing limitations and potential advancements charts a trajectory for reliable, weather-resilient autonomous transportation systems, ultimately contributing significantly to their integration into everyday life.