You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction (2205.14871v4)

Abstract: Challenging illumination conditions (low-light, under-exposure and over-exposure) in the real world not only cast an unpleasant visual appearance but also taint the computer vision tasks. After camera captures the raw-RGB data, it renders standard sRGB images with image signal processor (ISP). By decomposing ISP pipeline into local and global image components, we propose a lightweight fast Illumination Adaptive Transformer (IAT) to restore the normal lit sRGB image from either low-light or under/over-exposure conditions. Specifically, IAT uses attention queries to represent and adjust the ISP-related parameters such as colour correction, gamma correction. With only ~90k parameters and ~0.004s processing speed, our IAT consistently achieves superior performance over SOTA on the current benchmark low-light enhancement and exposure correction datasets. Competitive experimental performance also demonstrates that our IAT significantly enhances object detection and semantic segmentation tasks under various light conditions. Training code and pretrained model is available at https://github.com/cuiziteng/Illumination-Adaptive-Transformer.

• The paper presents a novel, lightweight transformer model—Illumination Adaptive Transformer (IAT)—designed to enhance image quality using only 90K parameters.
• It decomposes the ISP pipeline into a local branch for pixel-wise enhancements and a global branch for dynamic color and gamma corrections, boosting efficiency.
• Experiments demonstrate significant improvements in PSNR and SSIM along with rapid processing speeds, making IAT effective for low-light image and exposure corrections.

Analysis of "You Only Need 90K Parameters to Adapt Light: a Light Weight Transformer for Image Enhancement and Exposure Correction"

The paper "You Only Need 90K Parameters to Adapt Light: a Light Weight Transformer for Image Enhancement and Exposure Correction" presents a novel approach to enhance images taken under variable lighting conditions, aiming to restore visually pleasant and computationally valuable images from challenging illumination scenarios. The authors propose a compact and efficient model called the Illumination Adaptive Transformer (IAT), carefully designed to optimize the image signal processing (ISP) using only around 90,000 parameters.

Overview and Methodology

The primary contribution of this research is the decomposition of the ISP pipeline into local and global components, enabling an efficient lightweight model to handle image enhancement tasks. The proposed IAT model is noteworthy for its architectural design, which includes two specific branches:

1. Local Branch: This branch optimizes pixel-wise transformations, employing depth-wise convolution in place of the traditional attention mechanisms to enhance computational efficiency.
2. Global ISP Branch: Herein, the transformer-style architecture dynamically adjusts global ISP-related parameters, such as color and gamma correction. This branch leverages attention queries to manipulate and learn these parameters efficiently.

The separation into local and global components, combined with the applied transformations, enables the IAT to perform complex image enhancements with reduced computations, making it suitable for mobile or edge devices.

Results and Findings

The experimental results establish the superiority of the IAT over existing state-of-the-art methods across several benchmark datasets, including LOL (V1 and V2-real) for low-light enhancement, MIT-Adobe FiveK for general image enhancement, and a specialized exposure correction dataset. Notably, IAT outperforms in terms of both PSNR and SSIM metrics while drastically reducing model complexity and processing time to mere fractions of the demands posed by traditional methods.

Notable achievements of the IAT include:

• PSNR improvements reaching up to 23.5 on certain datasets.
• Processing speeds achieving 0.004 seconds per image, significantly faster than existing benchmarks.

The authors also report enhancements in object detection and semantic segmentation tasks when preprocessing with the IAT, underscoring its robustness and efficacy in upstream applications.

Implications and Future Directions

The development of the IAT opens up meaningful prospects for efficient image processing under a variety of lighting conditions. Its lightweight nature allows it to be readily applied in resource-constrained environments, broadening accessibility to advanced imaging technologies. The application of transformers in this manner may inspire further advancements in handling variable illumination within computer vision tasks.

Future research might delve into refining the ISP approximations within the model to address more intricate lighting and color challenges. Additionally, extending the integration of IAT with downstream AI workflows could bolster the tool's ubiquity across broader vision applications.

In conclusion, this paper advances the dialogue in image enhancement through a transformer-based approach and successfully aligns computational efficiency with enhanced performance in real-world conditions. The IAT exemplifies a significant leap in achieving high-quality image correction with minimalistic design, marking a progression toward more scalable vision solutions.