Low-Light Image and Video Enhancement Using Deep Learning: A Survey (2104.10729v3)

Abstract: Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination. Recent advances in this area are dominated by deep learning-based solutions, where many learning strategies, network structures, loss functions, training data, etc. have been employed. In this paper, we provide a comprehensive survey to cover various aspects ranging from algorithm taxonomy to open issues. To examine the generalization of existing methods, we propose a low-light image and video dataset, in which the images and videos are taken by different mobile phones' cameras under diverse illumination conditions. Besides, for the first time, we provide a unified online platform that covers many popular LLIE methods, of which the results can be produced through a user-friendly web interface. In addition to qualitative and quantitative evaluation of existing methods on publicly available and our proposed datasets, we also validate their performance in face detection in the dark.This survey together with the proposed dataset and online platform could serve as a reference source for future study and promote the development of this research field. The proposed platform and dataset as well as the collected methods, datasets, and evaluation metrics are publicly available and will be regularly updated.

• The paper surveys deep learning approaches for low-light enhancement, systematically categorizing supervised, unsupervised, and zero-shot methods.
• It highlights both empirical evaluations on benchmark datasets (e.g., LOL, SID) and computational strategies for real-time deployment.
• The survey identifies open challenges and future research directions, emphasizing scalability, robustness, and data diversity needs.

Low-Light Image and Video Enhancement Using Deep Learning: A Survey

The field of low-light image enhancement (LLIE) experiences significant challenges given the inherent issues associated with poorly illuminated imaging conditions. This survey paper offers an exhaustive overview of advancements in LLIE, emphasizing the growing prominence of deep learning solutions within this domain. The surveyed literature spans multiple methodologies and techniques, showcasing the progression from traditional methods towards modern deep learning-based approaches for both images and videos.

Methodological Innovations

The survey identifies several categories of deep learning-based LLIE techniques, including supervised, semi-supervised, unsupervised, reinforcement, and zero-shot learning. Supervised learning methods continue to dominate the field, largely due to well-known datasets such as LOL and SID, which provide paired image sets necessary for this approach. Techniques like the multi-branch architecture (MBLLEN) and Retinex-based methodologies (e.g., Retinex-Net, KinD) demonstrate that segregating the illumination and reflectance components yields promising results when enhancing low-light content.

Unsprervised learning, notably through frameworks like EnlightenGAN, bypasses the dependency on paired datasets, appealing for scenarios lacking comprehensive data sets. However, such techniques face instability and learning difficulties associated with the adversarial component. Zero-shot learning approaches like Zero-DCE and RUAS show promise in scenarios where external training pairs are unavailable, leveraging intrinsic image features for enhancement.

Empirical and Quantitative Evaluations

The paper introduces the comprehensive Low-Light Image and Video dataset acquired using a diverse range of mobile devices. This dataset provides a new, real-world benchmark for evaluating the generalization capability of LLIE models. Furthermore, an in-depth empirical analysis on various commonly used datasets like LOL, MIT-Adobe FiveK, and SID highlights the strengths and weaknesses of different methodologies in terms of different metrics. Important evaluation metrics include classical measures like PSNR and SSIM as well as perceptual-based metrics such as the LPIPS and non-reference evaluations such as NIQE.

Computational Considerations

The paper reviews the computational efficiency of network architectures, essential for real-time applications or deployment on limited-resource devices like smartphones. Although architectures such as U-Net are common, the community actively explores lightweight alternatives facilitating fast inference and deployment, evident in methods like Zero-DCE that adopt curve estimation rather than direct reconstruction.

Challenges and Future Research Directions

The researchers identify several open challenges in LLIE, ranging from algorithmic scalability and robustness to dataset creation. There's a compelling need for developing more realistic and larger-scale datasets, reflective of diverse real-world lighting conditions, and conducive to training more generalizable models. Additionally, crafting novel learning paradigms that effectively leverage the semantic information and preserve the perceptual naturalness of outputs remains an active research area.

In summary, this survey articulates the current landscape of deep learning solutions for LLIE, providing a valuable resource for researchers aiming to expand this field. The insights offered regarding comparative performances, unresolved challenges, and practical implications lay the groundwork for upcoming research tackling the nuances of both image and video enhancement under low-light conditions. Future work will likely explore the integration of deep learning models with more traditional methodologies, forming hybrid approaches capable of overcoming current limitations and advancing the robustness of LLIE solutions across diverse application scenarios.