Efficient Image Super-Resolution Using Pixel Attention (2010.01073v1)

Abstract: This work aims at designing a lightweight convolutional neural network for image super resolution (SR). With simplicity bare in mind, we construct a pretty concise and effective network with a newly proposed pixel attention scheme. Pixel attention (PA) is similar as channel attention and spatial attention in formulation. The difference is that PA produces 3D attention maps instead of a 1D attention vector or a 2D map. This attention scheme introduces fewer additional parameters but generates better SR results. On the basis of PA, we propose two building blocks for the main branch and the reconstruction branch, respectively. The first one - SC-PA block has the same structure as the Self-Calibrated convolution but with our PA layer. This block is much more efficient than conventional residual/dense blocks, for its twobranch architecture and attention scheme. While the second one - UPA block combines the nearest-neighbor upsampling, convolution and PA layers. It improves the final reconstruction quality with little parameter cost. Our final model- PAN could achieve similar performance as the lightweight networks - SRResNet and CARN, but with only 272K parameters (17.92% of SRResNet and 17.09% of CARN). The effectiveness of each proposed component is also validated by ablation study. The code is available at https://github.com/zhaohengyuan1/PAN.

• The paper introduces a pixel attention scheme that enhances feature extraction in image SR tasks with minimal computational overhead.
• It achieves competitive performance with significantly fewer parameters compared to models like SRResNet and CARN.
• The study validates innovative SC-PA and U-PA blocks through extensive experiments and ablation studies for lightweight, real-time applications.

Efficient Image Super-Resolution Using Pixel Attention

The paper presents an exploration into the optimization of convolutional neural networks (CNNs) for image super-resolution (SR) tasks, with a focus on minimizing computational overhead without compromising efficacy. This research introduces a novel pixel attention (PA) scheme, which is designed to enhance the feature extraction process inherent in SR models by producing three-dimensional attention maps. This differentiates PA from previous attention mechanisms such as channel attention (CA) and spatial attention (SA), which produce one-dimensional and two-dimensional maps respectively.

The proposed pixel attention network (PAN) manifests through two specialized blocks: the SC-PA block in the main branch and the U-PA block in the reconstruction branch. The SC-PA block integrates pixel attention into the self-calibrated convolution structure, optimizing efficiency relative to traditional residual and dense blocks. The U-PA block further reduces parameter requirements by employing nearest-neighbor upsampling in conjunction with convolutional and pixel attention layers. The PAN model achieves competitive performance to networks like SRResNet and CARN, with a significantly reduced parameter count of only 272K, translating to 17.92\% and 17.09\% of the parameter count of SRResNet and CARN, respectively.

Theoretical contributions are validated through extensive experimentation, showing that the novel PA leads to a notable improvement in SR performance, particularly within lightweight network design. Ablation studies further confirm the utility of PA, with enhancements observed in networks incorporating SC-PA and U-PA blocks. The implications for practical applications are significant, as the reduction in parameter size promotes real-time application feasibility in scenarios demanding high-definition SR, such as interactive image editing and video scaling.

In comparing with other state-of-the-art methods, PAN excels on certain datasets, and its minimal computational demand underscores its suitability for deployment in environments where resource constraints are a paramount concern. Furthermore, potential extensions of PA into larger network scales or other domains present an intriguing avenue for future research, particularly given the challenges identified with training larger models that incorporate PA without increasing computational complexity.

This research contributes to the ongoing discourse on balancing precision and computational efficiency in computer vision tasks, particularly within SR applications. The development of such light-weight yet effective models holds great promise for expanding the accessibility and practicality of high-performance image processing techniques across diverse technological ecosystem landscapes.