Deep Back-Projection Networks For Super-Resolution (1803.02735v1)

Abstract: The feed-forward architectures of recently proposed deep super-resolution networks learn representations of low-resolution inputs, and the non-linear mapping from those to high-resolution output. However, this approach does not fully address the mutual dependencies of low- and high-resolution images. We propose Deep Back-Projection Networks (DBPN), that exploit iterative up- and down-sampling layers, providing an error feedback mechanism for projection errors at each stage. We construct mutually-connected up- and down-sampling stages each of which represents different types of image degradation and high-resolution components. We show that extending this idea to allow concatenation of features across up- and down-sampling stages (Dense DBPN) allows us to reconstruct further improve super-resolution, yielding superior results and in particular establishing new state of the art results for large scaling factors such as 8x across multiple data sets.

• The paper introduces an innovative error feedback mechanism that iteratively refines the mapping from low-resolution to high-resolution images.
• It employs interlinked up- and down-sampling stages to effectively capture detailed features and preserve high-frequency information.
• Experimental results demonstrate significant PSNR and SSIM gains at large scaling factors, achieving superior performance with fewer parameters.

Deep Back-Projection Networks For Super-Resolution

The paper Deep Back-Projection Networks For Super-Resolution introduces a novel architecture aimed at advancing the state-of-the-art in single image super-resolution (SR). The proposed method, termed Deep Back-Projection Networks (DBPN), addresses fundamental limitations in traditional feed-forward deep neural networks by introducing an iterative up- and down-sampling mechanism designed to provide improved error feedback. This proposition is substantiated through methodical experimentation and analysis, establishing new performance benchmarks, especially for large scaling factors like $8\times$.

Key Contributions and Methodology

The paper emphasizes several pivotal contributions:

1. Error Feedback Mechanism: Unlike existing SR methods that implement a direct low-resolution (LR) to high-resolution (HR) transformation, DBPN employs an iterative error-correcting feedback loop. This loop involves upsampling to an intermediate HR space, downsampling it back to the LR space to compute an error map, and using this error map to guide the next upsampling iteration. This recursive process effectively minimizes cumulative projection errors at each stage.
2. Mutually Connected Up- and Down-Sampling Stages: The DBPN architecture alternates between up- and down-sampling stages, each designed with specific (de)convolution operators that project between multiple depth levels. This approach aims to capture intricate dependencies and rich features that are typically under-represented in direct mapping approaches.
3. Deep Concatenation: DBPN constructs detailed HR images by concatenating feature maps from all upsampling stages, thereby leveraging a variety of generated features. This process preserves essential high-frequency details that contribute to realistic and accurate SR outputs.
4. Dense Connection: An enhanced version of DBPN, termed Dense DBPN (D-DBPN), incorporates dense connectivity to improve feature reuse, gradient flow, and learning efficiency. Each stage in the dense version has access to the features from all preceding stages, further refining the reconstruction quality.

Experimental Results

The paper presents comprehensive experiments demonstrating the effectiveness of DBPN and its dense variant. Key findings include:

• Quantitative Metrics: DBPN and D-DBPN achieve superior PSNR and SSIM scores across multiple datasets (Set5, Set14, BSDS100, Urban100, Manga109) and scaling factors (2$\times$, 4$\times$, 8$\times$). Particularly, for the 8$\times$ scaling factor, DBPN outperforms existing state-of-the-art methods by a significant margin.
• Qualitative Analysis: Visual inspection of the SR images generated by DBPN reveals notable improvements in preserving fine textures and minimizing artifacts compared to baseline models. The iterative corrective feedback mechanism is especially effective in maintaining structural fidelity for high scaling factors.
• Performance vs. Parameters: DBPN demonstrates a favorable trade-off between performance and computational cost. Notably, DBPN requires fewer parameters compared to methods like EDSR while achieving comparable or superior results, highlighting its efficiency and scalability.

Implications and Future Work

The introduction of DBPN and D-DBPN has several practical and theoretical implications:

• Practical Applications: By enhancing the quality and reliability of SR algorithms, the techniques proposed could be pivotal in applications requiring high-fidelity image reconstruction, such as medical imaging, remote sensing, and detail enhancement in consumer electronics.
• Theoretical Insights: The paper underscores the importance of iterative feedback in neural network design for ill-posed inverse problems like SR. The demonstrated efficacy of up- and down-sampling stages with error correction encourages further exploration into recursive and feedback-driven learning models.
• Future Directions: The modular nature of DBPN opens avenues for integrating this approach with other advanced techniques, such as adversarial training with GANs to further improve perceptual quality. Exploring the balance between dense connections and depth could also optimize both computational efficiency and SR accuracy.

In conclusion, the Deep Back-Projection Networks framework represents a substantial advancement in the SR field by effectively addressing representation limitations and enhancing error feedback using an iterative processing strategy. The robust experimental validation and promising results position DBPN and its dense variant as a compelling choice for future research and application in SR and related domains.