LLVIP: A Visible-infrared Paired Dataset for Low-light Vision (2108.10831v4)

Abstract: It is very challenging for various visual tasks such as image fusion, pedestrian detection and image-to-image translation in low light conditions due to the loss of effective target areas. In this case, infrared and visible images can be used together to provide both rich detail information and effective target areas. In this paper, we present LLVIP, a visible-infrared paired dataset for low-light vision. This dataset contains 30976 images, or 15488 pairs, most of which were taken at very dark scenes, and all of the images are strictly aligned in time and space. Pedestrians in the dataset are labeled. We compare the dataset with other visible-infrared datasets and evaluate the performance of some popular visual algorithms including image fusion, pedestrian detection and image-to-image translation on the dataset. The experimental results demonstrate the complementary effect of fusion on image information, and find the deficiency of existing algorithms of the three visual tasks in very low-light conditions. We believe the LLVIP dataset will contribute to the community of computer vision by promoting image fusion, pedestrian detection and image-to-image translation in very low-light applications. The dataset is being released in https://bupt-ai-cz.github.io/LLVIP. Raw data is also provided for further research such as image registration.

• The paper introduces LLVIP, a synchronized dataset of 15,488 visible-infrared image pairs that advances research in low-light vision challenges.
• It evaluates state-of-the-art methods such as DenseFuse, IFCNN, Yolov5, and Pix2Pix GAN to reveal strengths and limitations in image fusion and pedestrian detection.
• Findings highlight the need for enhanced multispectral algorithms to improve detail preservation and detection accuracy under extreme low-light conditions.

LLVIP: A Visible-infrared Paired Dataset for Low-light Vision

The paper introduces LLVIP, a significant contribution to the field of computer vision through the introduction of a visible-infrared paired dataset specifically crafted for low-light vision applications. This dataset, comprising 15,488 rigorously aligned image pairs complete with pedestrian annotations, aims to enhance research and development in challenging low-light visual tasks such as image fusion, pedestrian detection, and image-to-image translation.

Dataset Characteristics and Justification

The LLVIP dataset fills a crucial gap by providing a synchronous and high-resolution visible-infrared image dataset subjected to extreme low-light conditions. The significance of combining infrared and visible imagery is underscored by the complementary nature of these modalities: while visible images capture rich textures, infrared images offer robustness in target detection despite low visibility in the visible spectrum.

Evaluation of Methods on LLVIP

The paper evaluates a range of state-of-the-art visual algorithms on this dataset, encompassing image fusion algorithms such as DenseFuse and IFCNN, pedestrian detection using Yolov5 and Yolov3, and image-to-image translation exemplified by Pix2Pix GAN. The results elucidate the complementary benefits and deficiencies in current methodologies:

1. Image Fusion: Through subjective and objective assessments, DenseFuse ($l_1$ strategy) and IFCNN emerge as relatively proficient in balancing detail preservation in poor light by combining visual cues from both modalities. Nonetheless, issues such as detail loss in dark regions indicate substantial room for improvement in fusion strategies.
2. Pedestrian Detection: Experiments reveal that pedestrian detection using traditional visible light datasets struggles significantly in low-light scenarios, as reflected by lower AP scores compared to infrared-based detection that performs better under these conditions. This finding highlights the necessity to leverage multispectral data for improved detection accuracy in suboptimal lighting.
3. Image-to-Image Translation: The translation from visible to infrared demonstrates less than satisfactory results when assessed using metrics such as SSIM and PSNR, indicating that algorithms such as Pix2Pix GAN face significant challenges in modeling complex thermal imaging correlations in diverse conditions in LLVIP.

Implications and Future Directions

The LLVIP dataset proposes a benchmark that not only challenges existing algorithms considerably but also motivates further exploration into more efficient multispectral algorithms capable of bridging performance gaps. The alignment in time and space alongside carefully annotated pedestrian data positions LLVIP as a critical resource in:

• Developing robust low-light pedestrian detection systems by integrating multispectral inputs.
• Advancing image fusion techniques tailored to nighttime visibility and surveillance requirements.
• Pioneering novel image translation models that effectively extrapolate thermal features from visible imagery.

Long-term, LLVIP fosters a pathway towards refining algorithmic approaches and deepening theoretical understanding of multispectral data synergy under adverse lighting conditions. Future developments can include exploring advanced methods for enhanced feature representation, possibly incorporating newer architectures in unsupervised and semi-supervised learning, and testing novel sensor fusion approaches to realize truly adaptive, real-world capable vision systems.