Residual Non-local Attention Networks for Image Restoration (1903.10082v1)

Abstract: In this paper, we propose a residual non-local attention network for high-quality image restoration. Without considering the uneven distribution of information in the corrupted images, previous methods are restricted by local convolutional operation and equal treatment of spatial- and channel-wise features. To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts. Specifically, we design trunk branch and (non-)local mask branch in each (non-)local attention block. The trunk branch is used to extract hierarchical features. Local and non-local mask branches aim to adaptively rescale these hierarchical features with mixed attentions. The local mask branch concentrates on more local structures with convolutional operations, while non-local attention considers more about long-range dependencies in the whole feature map. Furthermore, we propose residual local and non-local attention learning to train the very deep network, which further enhance the representation ability of the network. Our proposed method can be generalized for various image restoration applications, such as image denoising, demosaicing, compression artifacts reduction, and super-resolution. Experiments demonstrate that our method obtains comparable or better results compared with recently leading methods quantitatively and visually.

• The paper introduces the RNAN architecture that integrates residual local and non-local attention blocks for adaptive feature extraction in image restoration.
• It leverages a novel non-local attention mechanism to capture long-range dependencies, enhancing performance on challenging image denoising and artifact reduction tasks.
• Numerical results show state-of-the-art improvements in PSNR and SSIM, underscoring the network’s robustness and efficiency.

Residual Non-local Attention Networks for Image Restoration

The paper "Residual Non-local Attention Networks for Image Restoration" introduces the Residual Non-local Attention Network (RNAN), which is designed to enhance image restoration tasks such as denoising, demosaicing, compression artifacts reduction, and super-resolution. The authors propose a novel approach that addresses the limitations of existing Convolutional Neural Networks (CNNs) by incorporating both local and non-local attention mechanisms to capture long-range dependencies in images.

Key Contributions

1. Architecture Design:

The RNAN architecture integrates residual local and non-local attention blocks that consist of trunk and mask branches. This design facilitates the extraction of hierarchical features and the application of attention mechanisms to selectively focus on specific regions of the image. The trunk branch extracts hierarchical features, while the mask branch generates attention maps to rescale these features adaptively.

2. Non-local Attention Mechanism:

The incorporation of non-local blocks within the mask branch enables the network to collect information from the entire image, thus enhancing its ability to deal with intricate image restoration tasks. This approach is particularly beneficial for images with severe corruption, where long-range dependency is crucial.

3. Residual Non-local Attention Learning:

The authors introduce a learning strategy that involves preserving low-level features through residual learning, which suits image restoration tasks more than traditional high-level vision tasks. This strategy helps maintain critical features that contribute to the image quality.

Numerical Results

The RNAN achieves state-of-the-art results across several benchmark datasets. For instance, in color image denoising tasks, the RNAN surpasses other leading methods in metrics like PSNR and SSIM, particularly at higher noise levels where capturing global context proves beneficial. For image compression artifact reduction, the network exhibits superior performance even under low-quality image conditions, showcasing its robustness and efficacy.

Implications and Future Directions

The novel incorporation of non-local attention in image restoration tasks has significant theoretical and practical implications. Practically, the RNAN's ability to improve image quality across various tasks with moderate model size and efficiency is notable. Theoretically, this advancement opens pathways for exploring more complex architectures that leverage long-range dependencies and contextual information.

Future work can extend these concepts to tackle other challenges in computer vision or refine the attention mechanisms to further enhance specificity and performance. The integration of RNANs in real-time applications where image quality is paramount could transform industries reliant on high-quality visual data.

In summary, the paper offers a compelling advancement in the field of image processing through the innovative use of residual non-local attention networks, setting a new benchmark for image restoration methodologies.