Dynamic Attentive Graph Learning for Image Restoration (2109.06620v1)

Abstract: Non-local self-similarity in natural images has been verified to be an effective prior for image restoration. However, most existing deep non-local methods assign a fixed number of neighbors for each query item, neglecting the dynamics of non-local correlations. Moreover, the non-local correlations are usually based on pixels, prone to be biased due to image degradation. To rectify these weaknesses, in this paper, we propose a dynamic attentive graph learning model (DAGL) to explore the dynamic non-local property on patch level for image restoration. Specifically, we propose an improved graph model to perform patch-wise graph convolution with a dynamic and adaptive number of neighbors for each node. In this way, image content can adaptively balance over-smooth and over-sharp artifacts through the number of its connected neighbors, and the patch-wise non-local correlations can enhance the message passing process. Experimental results on various image restoration tasks: synthetic image denoising, real image denoising, image demosaicing, and compression artifact reduction show that our DAGL can produce state-of-the-art results with superior accuracy and visual quality. The source code is available at https://github.com/jianzhangcs/DAGL.

• The paper proposes DAGL, a dynamic graph-based model that adapts patch-level non-local correlations for superior image restoration.
• It employs a Feature Extraction Module and a Graph-based Feature Aggregation Module with differentiable KNN for adaptable neighbor selection.
• Experiments demonstrate that DAGL outperforms existing methods in both synthetic and real-image denoising, achieving higher PSNR and SSIM scores.

Dynamic Attentive Graph Learning for Image Restoration

The paper "Dynamic Attentive Graph Learning for Image Restoration" introduces a novel approach for image restoration by building upon the principles of non-local self-similarity and the dynamic nature of graph-based learning models. The proposed model, Dynamic Attentive Graph Learning (DAGL), innovatively utilizes a graph framework to dynamically adapt non-local correlations at the patch level rather than at a fixed scale, as is common in many existing methods. This paper presents significant contributions in enhancing image restoration tasks, which include synthetic and real image denoising, image demosaicing, and compression artifact reduction.

Methodology and Model Architecture

The authors propose a model that intelligently constructs graphs where nodes represent feature patches rather than individual pixels. This distinguishes DAGL from the conventional non-local methods where pixel-based correlations can be easily distorted by image degradation. The model's architecture consists of two core components: a Feature Extraction Module (FEM) and the Graph-based Feature Aggregation Module (GFAM). FEM employs residual blocks to obtain deep feature embeddings, which are further processed by GFAM. The innovation in GFAM is its ability to adaptively determine the number of neighbors or graph connections for each node, addressing the variation in image content and degradation.

This dynamic connection allocation is facilitated through a differentiable K-Nearest Neighbors (KNN) module integrated within the model, enabling a flexible approach for constructing graph-based representations. The focus on adaptive neighbor selection is a response to the understanding that different image content demands varying levels of non-local correlation for optimal restoration.

Experimental Results

The authors conducted extensive experiments across different image restoration tasks, showcasing the efficacy of DAGL:

• Synthetic Image Denoising: On datasets like Urban100, BSD68, and Set12, DAGL consistently achieved superior PSNR and SSIM scores compared to established methods like BM3D, DnCNN, and NLRN. For instance, in Urban100, at noise level σ=25, DAGL achieved a PSNR of 31.39, outperforming other state-of-the-art methods.
• Real Image Denoising: When evaluated on the DND dataset, DAGL achieved high PSNR and SSIM scores (39.83 and 0.957, respectively), surpassing recent methods like CycleISP and GDANet. This emphasizes its robustness to realistic noise.
• Image Demosaicing and Compression Artifact Reduction: DAGL maintained strong performance, outperforming methods like RNAN with improved visual and quantitative results on datasets such as McMaster18 and LIVE1.

Implications and Future Directions

The capability of DAGL to adaptively optimize graph connections has significant implications for advancing AI applications in image restoration. Practically, this method can be extended to other tasks that benefit from non-local dependencies, such as video processing and medical imaging, where maintaining fine details is crucial. Theoretically, the adaptive nature of DAGL suggests potential enhancements in graph neural networks, particularly in domains requiring pixel or patch-level transformations.

The advancement presented in DAGL also opens the door to exploring further improvements in computational efficiency and scalability, particularly with its application on large-scale datasets or real-time systems. Future research might focus on integrating similar dynamic graph approaches in other neural network architectures, thereby broadening the scope and improving the adaptability of machine learning models in numerous practical applications.