HIT-UAV: A high-altitude infrared thermal dataset for Unmanned Aerial Vehicle-based object detection (2204.03245v2)

Abstract: We present the HIT-UAV dataset, a high-altitude infrared thermal dataset for object detection applications on Unmanned Aerial Vehicles (UAVs). The dataset comprises 2,898 infrared thermal images extracted from 43,470 frames in hundreds of videos captured by UAVs in various scenarios including schools, parking lots, roads, and playgrounds. Moreover, the HIT-UAV provides essential flight data for each image, such as flight altitude, camera perspective, date, and daylight intensity. For each image, we have manually annotated object instances with bounding boxes of two types (oriented and standard) to tackle the challenge of significant overlap of object instances in aerial images. To the best of our knowledge, the HIT-UAV is the first publicly available high-altitude UAV-based infrared thermal dataset for detecting persons and vehicles. We have trained and evaluated well-established object detection algorithms on the HIT-UAV. Our results demonstrate that the detection algorithms perform exceptionally well on the HIT-UAV compared to visual light datasets since infrared thermal images do not contain significant irrelevant information about objects. We believe that the HIT-UAV will contribute to various UAV-based applications and researches. The dataset is freely available at https://github.com/suojiashun/HIT-UAV-Infrared-Thermal-Dataset.

HIT-UAV: A High-Altitude Infrared Thermal Dataset for UAV-Based Object Detection

The paper "HIT-UAV: A high-altitude infrared thermal dataset for Unmanned Aerial Vehicle-based object detection" introduces the HIT-UAV dataset, an innovative collection of high-altitude infrared thermal images specifically aimed at enhancing object detection applications using UAVs. The dataset contains 2,898 infrared thermal images, which are derived from a substantial volume of 43,470 video frames captured in various real-world scenarios such as schools, parking lots, roads, and playgrounds. Critically, each image in the dataset is accompanied by essential UAV flight data, including flight altitude and camera perspective, enhancing the utility of the dataset for research purposes.

Characteristics and Annotations

The HIT-UAV dataset offers comprehensive annotations for each image, detailing object instances with both oriented and standard bounding boxes. The inclusion of oriented bounding boxes seeks to address and mitigate the prevalent issue of significant overlap found in aerial images, which can complicate object detection tasks. The dataset targets five object categories: Person, Car, Bicycle, OtherVehicle, and DontCare, with the latter category encompassing objects that are difficult to classify precisely. Notably, the average number of object bounding boxes per image in the HIT-UAV dataset, comparing favorably to established datasets like PASCAL VOC and MSCOCO, highlights its richness in terms of object density.

Dataset and Research Implications

The HIT-UAV dataset fills a specific niche not previously addressed by existing datasets, which tend to either focus on visual light images or low-altitude thermal images. By providing a dataset that effectively utilizes infrared thermal imagery at high altitudes, the authors provide a robust tool for enhancing UAV-based detection capabilities, particularly in scenarios with limited lighting—such as night-time operations. Infrared imagery's proficiency in filtering irrelevant visual clutter aids in achieving superior detection results, as demonstrated by experimental evaluations in the paper.

Object Detection Performance

The paper includes an empirical evaluation of several well-established object detection algorithms, including YOLOv4, YOLOv4-tiny, Faster R-CNN, and SSD, trained on the HIT-UAV dataset. Results show that infrared thermal images enable these models to perform exceptionally well, notably surpassing some visual light image datasets in detection accuracy. Specifically, YOLOv4 achieves a mean Average Precision (mAP) of 84.75% on the HIT-UAV test set, underscoring the dataset's capability to facilitate high-precision object detection in UAV applications.

Future Research Directions

The development and availability of the HIT-UAV dataset are expected to foster numerous research avenues, such as the exploration of infrared thermal camera applicability in UAV-based search and rescue missions, particularly during night operations. Furthermore, the dataset allows for detailed analysis of how UAV flight parameters, like altitude and camera perspective, impact object detection effectiveness. This could lead to improvements in flight strategies and detection algorithms, adapting more flexibly to varied operational scenarios.

Conclusion

The HIT-UAV dataset serves as a valuable resource for advancing the state of UAV-based object detection systems, particularly by utilizing high-altitude infrared thermal images. This dataset not only bridges existing gaps in UAV-based object detection but also stimulates further research in optimizing UAV strategies for complex operational environments. The insights drawn from experiments with this dataset could translate into practical enhancements in UAV deployments for surveillance, reconnaissance, and emergency response purposes.