LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models (2407.08939v1)

Abstract: In this paper, we propose a diffusion-based unsupervised framework that incorporates physically explainable Retinex theory with diffusion models for low-light image enhancement, named LightenDiffusion. Specifically, we present a content-transfer decomposition network that performs Retinex decomposition within the latent space instead of image space as in previous approaches, enabling the encoded features of unpaired low-light and normal-light images to be decomposed into content-rich reflectance maps and content-free illumination maps. Subsequently, the reflectance map of the low-light image and the illumination map of the normal-light image are taken as input to the diffusion model for unsupervised restoration with the guidance of the low-light feature, where a self-constrained consistency loss is further proposed to eliminate the interference of normal-light content on the restored results to improve overall visual quality. Extensive experiments on publicly available real-world benchmarks show that the proposed LightenDiffusion outperforms state-of-the-art unsupervised competitors and is comparable to supervised methods while being more generalizable to various scenes. Our code is available at https://github.com/JianghaiSCU/LightenDiffusion.

An Overview of LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models

The paper "LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models" presents a novel unsupervised framework for enhancing low-light images by integrating Retinex theory with diffusion models. The authors propose an innovative approach to address the challenges associated with low-light image enhancement (LLIE), such as limited visibility and noise, by leveraging the strengths of both physically explainable theories and advanced generative models.

Core Contributions

The primary contribution of this paper is the introduction of LightenDiffusion, a framework that effectively combines Retinex decomposition with diffusion models to achieve unsupervised LLIE. The authors propose several key components within the framework:

1. Content-Transfer Decomposition Network (CTDN): This network performs Retinex decomposition in the latent space rather than the image space. By encoding the features of low-light and normal-light images, the CTDN effectively separates content-rich reflectance maps from content-free illumination maps. This separation in latent space enhances the accuracy and efficacy of the subsequent enhancement process, improving visual quality significantly over traditional Retinex-based methods that operate in the image space.
2. Latent-Retinex Diffusion Model (LRDM): The LRDM leverages the decomposition results from the CTDN to guide a diffusion model in transforming unpaired low-light inputs into enhanced images. By utilizing the generative capabilities of diffusion models, the LRDM addresses potential artifacts and content loss issues that typically occur with decomposition-based methods.
3. Self-Constrained Consistency Loss: To ensure the output retains the inherent content of the low-light input, the authors introduce a novel loss function that enforces consistency in the intrinsic structure between the input and enhanced output. This mechanism allows the diffusion model to perform denoising and enhancement effectively.

Experimental Validation

The efficacy and generalizability of LightenDiffusion are supported by extensive experiments conducted on standard datasets. The method outperforms existing unsupervised approaches and is competitive with supervised methods, showcasing its robustness in diverse real-world scenarios. Quantitative metrics like PSNR and SSIM indicate superior enhancement quality, while qualitative evaluations reveal visually pleasing results that maintain detail fidelity without introducing artifacts. Additionally, the practical relevance of LightenDiffusion is demonstrated through improved performance in low-light face detection tasks, highlighting its potential utility in enhancing downstream computer vision applications.

Implications and Speculation

The implications of LightenDiffusion extend both practically and theoretically. Practically, the framework provides a robust tool for LLIE that can be utilized across various applications, such as photography and surveillance, where lighting conditions often vary. Theoretically, the novel integration of Retinex theory with diffusion models establishes a pathway for future work in combining explainable AI concepts with complex generative models. This approach may inspire further exploration into hybrid models that synergize the strengths of both worlds to tackle other challenging image processing tasks.

Looking forward, future developments can pivot towards refining the model's efficiency and adaptability to broader applications beyond low-light scenarios, potentially expanding it into domains demanding high-fidelity image restoration under a range of degradations. Furthermore, exploiting additional unlabeled datasets or synthetic data could accelerate the unsupervised learning process within similar frameworks, leveraging the growing availability of diverse image data.

In conclusion, LightenDiffusion represents a significant step toward practical and effective unsupervised image enhancement, offering a compelling blend of theory-driven and data-driven techniques for advancing the state of the art in low-light image processing.