LYT-NET: Lightweight YUV Transformer-based Network for Low-light Image Enhancement

Abstract: This letter introduces LYT-Net, a novel lightweight transformer-based model for low-light image enhancement (LLIE). LYT-Net consists of several layers and detachable blocks, including our novel blocks--Channel-Wise Denoiser (CWD) and Multi-Stage Squeeze & Excite Fusion (MSEF)--along with the traditional Transformer block, Multi-Headed Self-Attention (MHSA). In our method we adopt a dual-path approach, treating chrominance channels U and V and luminance channel Y as separate entities to help the model better handle illumination adjustment and corruption restoration. Our comprehensive evaluation on established LLIE datasets demonstrates that, despite its low complexity, our model outperforms recent LLIE methods. The source code and pre-trained models are available at https://github.com/albrateanu/LYT-Net

• The paper presents a transformer-based network that leverages the YUV color space for targeted low-light image enhancement.
• It introduces detachable modules like the Channel-wise Denoiser and Multi-stage Squeeze and Excite Fusion to balance noise reduction and feature extraction.
• Experimental results show competitive PSNR and SSIM with reduced FLOPS (3.49G) and minimal parameters (0.045M), enabling efficient deployment.

Overview of LYT-NET: LIGHTWEIGHT YUV TRANSFORMER-BASED NETWORK FOR LOW-LIGHT IMAGE ENHANCEMENT

This paper presents LYT-Net, an innovative approach for enhancing low-light images, a challenging task in computer vision. Developed by an interdisciplinary research team, LYT-Net leverages a transformer-based architecture to process images in the YUV color space, distinguishing itself from traditional Retinex-based models and direct CNN mappings. This strategy uniquely exploits the separation of luminance (Y) and chrominance (U and V) to achieve a fine-grained balance between light and color enhancement. The design objective of LYT-Net prioritizes computational efficiency without compromising the quality of enhancement, a significant consideration in real-world applications where resource constraints are prevalent.

Methodological Advances

The architecture of LYT-Net is characterized by several key components:

• YUV Color Space Utilization: By processing images in the YUV color space, LYT-Net achieves separate handling of luminance and chrominance. This separation allows for targeted enhancement, wherein the model enhances luminance for improved visibility without distorting color information, performing particularly well on human-perceptual levels.
• Transformer-Based Architecture: The inclusion of a Multi-headed Self-attention (MHSA) Block within the model architecture optimizes the capture of long-range dependencies, fundamental for comprehending comprehensive image contexts in low-light conditions. This mechanism allows the model architecture to focus on spatial variability critical for LLIE.
• Detachable Blocks: LYT-Net employs innovative blocks such as the Channel-wise Denoiser (CWD) and Multi-stage Squeeze and Excite Fusion (MSEF), which integrate convolutional and attention-based operations, balancing feature extraction and noise reduction.
• Hybrid Loss Function: The training process employs a multifaceted loss function incorporating Smooth L1 loss, Perceptual loss, Histogram loss, PSNR loss, Color loss, and MS-SSIM loss, addressing diverse enhancement criteria while efficiently guiding model convergence.

Experimental Validation

The empirical evaluation of LYT-Net utilizes datasets including LOL-v1, LOL-v2-real, and LOL-v2-synthetic, benchmarking its performance against state-of-the-art models. Quantitative assessments reveal that LYT-Net is on par with, or outperforms, other techniques in terms of PSNR and SSIM metrics. Notably, it achieves third-best complexity with significantly reduced FLOPS (3.49G) and parameter counts (0.045M), underscoring its suitability for deployment in environments where computational efficiency is essential.

Qualitative assessments further corroborate these findings. Visual comparisons reveal that LYT-Net excels in balancing luminance enhancement and color fidelity, delivering superior enhancements without over- or under-exposure, a common issue with contemporary methods.

Implications and Future Directions

LYT-Net’s design leverages the optimal separation of color components inherent in the YUV space, aiming to set a new standard for LLIE by demonstrating the potential of lightweight solutions. Its deployment could be transformative in areas like mobile imaging, video surveillance, and autonomous systems where operating under varying lighting conditions is a practical challenge.

Theoretically, the paper invites further exploration of transformer-based architectures in image enhancement domains. It posits the potential extensibility of such models beyond LLIE, such as in high dynamic range (HDR) imaging and other image restoration tasks.

The authors suggest interesting avenues for future exploration, including expanding the training and testing datasets to refine model robustness further. Additionally, given LYT-Net’s efficiency, potential integration with sensor technology signals opportunities for real-time applications, broadening its impact across diverse technological realms.

In conclusion, while LYT-Net proposes a compact and power-efficient framework for resolving LLIE challenges, it also exemplifies the broader applicability of transformer-based models in advancing image processing capabilities in resource-constrained environments.