Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement (2303.06705v3)

Abstract: When enhancing low-light images, many deep learning algorithms are based on the Retinex theory. However, the Retinex model does not consider the corruptions hidden in the dark or introduced by the light-up process. Besides, these methods usually require a tedious multi-stage training pipeline and rely on convolutional neural networks, showing limitations in capturing long-range dependencies. In this paper, we formulate a simple yet principled One-stage Retinex-based Framework (ORF). ORF first estimates the illumination information to light up the low-light image and then restores the corruption to produce the enhanced image. We design an Illumination-Guided Transformer (IGT) that utilizes illumination representations to direct the modeling of non-local interactions of regions with different lighting conditions. By plugging IGT into ORF, we obtain our algorithm, Retinexformer. Comprehensive quantitative and qualitative experiments demonstrate that our Retinexformer significantly outperforms state-of-the-art methods on thirteen benchmarks. The user study and application on low-light object detection also reveal the latent practical values of our method. Code, models, and results are available at https://github.com/caiyuanhao1998/Retinexformer

• The paper introduces a one-stage model that integrates Transformer and Retinex theory for low-light image enhancement.
• It employs an innovative Illumination-Guided Multi-head Self-Attention mechanism to capture long-range dependencies in varying lighting.
• Experimental results show over 6 dB improvement on benchmark datasets, demonstrating significant advancements over state-of-the-art methods.

Retinexformer: One-Stage Retinex-Based Transformer for Low-Light Image Enhancement

The paper presents a novel approach for low-light image enhancement through the development of Retinexformer, an algorithm that leverages the principles of Retinex theory and the computational power of Transformers. Traditional methods for low-light enhancement have typically relied on multi-stage training processes and convolutional neural networks (CNNs), which often struggle with capturing long-range dependencies. This research introduces a groundbreaking shift by employing a Transformer-based framework to achieve more effective image enhancement in a single stage.

The core of the proposed method is the One-stage Retinex-based Framework (ORF), designed to address the limitations posed by existing algorithms. ORF operates by first estimating the illumination information required to 'light up' the low-light image. Subsequently, it utilizes a corruption restoration process to rectify any noise or color distortions present in the enhanced image. This approach is not only computationally efficient but also effectively addresses the corruptions that may arise during enhancement, which are often ignored by conventional methods.

A key innovation of this work is the integration of the Illumination-Guided Transformer (IGT) within ORF. The IGT employs an Illumination-Guided Multi-head Self-Attention (IG-MSA) mechanism, which is pivotal in modeling interactions between regions of varying lighting conditions. The IG-MSA method makes use of illumination representations to guide the computation of self-attention, thereby facilitating robust modeling of long-range dependencies.

The results from comprehensive experiments demonstrate that Retinexformer significantly outperforms a broad range of state-of-the-art methods on thirteen benchmark datasets. Notably, the improvement in performance exceeds 6 dB on the SID and SDSD-indoor datasets, underscoring the strengths of this approach in enhancing image quality in challenging low-light conditions.

The paper provides extensive quantitative and qualitative evaluations, including user studies and low-light object detection tasks, which indicate promising practical applications of the method. These results suggest that Retinexformer not only improves visual perception in low-light images but also enhances downstream tasks in computer vision.

From a theoretical perspective, Retinexformer reflects a successful adaptation of Transformers to low-light image enhancement, an area traditionally dominated by CNN-based methods. This transformation demonstrates the potential of Transformers to handle complex image processing tasks by effectively modeling global dependencies in image data.

Looking forward, this research opens avenues for further exploration into Transformer-based methods for various image enhancement tasks. Potential future work could explore adapting this framework to other types of image degradation and expanding its application in real-time scenarios, potentially transforming how image processing algorithms address visibility issues in low-light conditions. The development of Retinexformer signifies a meaningful step in advancing low-light image enhancement technologies, with implications for both academic research and practical applications in various industries.