Residual Feature Distillation Network for Lightweight Image Super-Resolution (2009.11551v1)

Abstract: Recent advances in single image super-resolution (SISR) explored the power of convolutional neural network (CNN) to achieve a better performance. Despite the great success of CNN-based methods, it is not easy to apply these methods to edge devices due to the requirement of heavy computation. To solve this problem, various fast and lightweight CNN models have been proposed. The information distillation network is one of the state-of-the-art methods, which adopts the channel splitting operation to extract distilled features. However, it is not clear enough how this operation helps in the design of efficient SISR models. In this paper, we propose the feature distillation connection (FDC) that is functionally equivalent to the channel splitting operation while being more lightweight and flexible. Thanks to FDC, we can rethink the information multi-distillation network (IMDN) and propose a lightweight and accurate SISR model called residual feature distillation network (RFDN). RFDN uses multiple feature distillation connections to learn more discriminative feature representations. We also propose a shallow residual block (SRB) as the main building block of RFDN so that the network can benefit most from residual learning while still being lightweight enough. Extensive experimental results show that the proposed RFDN achieve a better trade-off against the state-of-the-art methods in terms of performance and model complexity. Moreover, we propose an enhanced RFDN (E-RFDN) and won the first place in the AIM 2020 efficient super-resolution challenge. Code will be available at https://github.com/njulj/RFDN.

Residual Feature Distillation Network for Lightweight Image Super-Resolution

The paper, authored by Jie Liu, Jie Tang, and Gangshan Wu, introduces an innovative approach to single image super-resolution (SISR) with their Residual Feature Distillation Network (RFDN). This work addresses the persistent challenge of balancing high-performance SISR with computational efficiency, crucial for deployment on edge devices.

Core Contributions

The paper makes several important contributions to the field:

1. Feature Distillation Connection (FDC): The authors propose FDC as an efficient alternative to the channel splitting operation used in the Information Distillation Network (IDN). FDC offers a method to separate and refine features with minimal computational overhead.
2. Shallow Residual Block (SRB): The SRB is introduced as a lightweight building block that leverages residual learning to enhance feature representations without substantial increase in parameters.
3. Residual Feature Distillation Network (RFDN): By integrating multiple FDCs and SRBs, the authors construct RFDN, a model that achieves high performance with lower complexity compared to existing methods like IDN and IMDN.
4. Extensive Evaluation: The paper presents extensive experimental results showing that RFDN performs competitively with state-of-the-art models in terms of both PSNR and model size, achieving a good trade-off between performance and computational cost.
5. Enhanced Model Performance: By proposing enhanced RFDN (E-RFDN), the authors secure first place in the AIM 2020 efficient super-resolution challenge, demonstrating the practical efficiency and adaptability of their method.

Methodology and Insights

The authors provide substantial insight into the information distillation mechanism, rethinking and redesigning it for better efficiency. The re-evaluation of the channel splitting strategy has led to the development of the FDC, allowing simultaneous refinement and retention of feature stems using lighter convolutional operations.

The SRB further complements this approach by introducing identity mappings, ensuring that the network benefits from residual learning without an increase in model complexity. The combination of these components within RFDN enhances super-resolution performance while maintaining a reduced parameter count and computational footprint.

The research evidence suggests that FDC, when combined with SRB, provides a significant boost in SR performance due to improved feature learning and representation. The architecture's consideration of computational constraints makes it viable for deployment in resource-limited scenarios, such as real-time video streaming on mobile devices.

Numerical and Comparative Results

RFDN outperforms numerous lightweight SR models, such as MemNet and IDN, in terms of both PSNR and model parameters. Notably, the enhanced version, E-RFDN, showcases its efficiency by winning the AIM 2020 challenge, offering not only superior runtime performance but also minimal resource consumption in comparison to other competitors.

Implications and Future Directions

The findings imply significant improvements in the design of lightweight SR models, particularly in the context of mobile and embedded systems. The introduction of FDC and SRB could influence the development of future models that require balancing computational budget with high-resolution output quality.

Future research could explore further optimization of the FDC and SRB within different network architectures or even extend the approach to other image processing tasks. Adaptation to tasks involving dynamic scene understanding could be another avenue, leveraging the efficiency and performance demonstrated by RFDN.

In conclusion, the paper presents a compelling contribution to the SISR domain by reducing the computational demands of high-performance SR networks and paving the way for their real-world applicability across various platforms.