Towards Compact Single Image Super-Resolution via Contrastive Self-distillation (2105.11683v1)

Abstract: Convolutional neural networks (CNNs) are highly successful for super-resolution (SR) but often require sophisticated architectures with heavy memory cost and computational overhead, significantly restricts their practical deployments on resource-limited devices. In this paper, we proposed a novel contrastive self-distillation (CSD) framework to simultaneously compress and accelerate various off-the-shelf SR models. In particular, a channel-splitting super-resolution network can first be constructed from a target teacher network as a compact student network. Then, we propose a novel contrastive loss to improve the quality of SR images and PSNR/SSIM via explicit knowledge transfer. Extensive experiments demonstrate that the proposed CSD scheme effectively compresses and accelerates several standard SR models such as EDSR, RCAN and CARN. Code is available at https://github.com/Booooooooooo/CSD.

• The paper presents a novel contrastive self-distillation framework that compresses SISR models without compromising reconstruction quality.
• It utilizes a Channel-Splitting Super-Resolution network (CSSR-Net) that selectively transfers knowledge from a teacher network to reduce parameters and support dynamic inference.
• Experiments reveal a 4× compression rate and 1.77× speedup with minimal loss in PSNR and SSIM, paving the way for deployment on resource-limited devices.

Compact Single Image Super-Resolution via Contrastive Self-distillation

This paper presents an innovative approach to Single Image Super-Resolution (SISR) that addresses the fundamental challenge of deploying convolutional neural network (CNN) models on resource-constrained devices. Traditional SISR models, although successful in restoring high-resolution images from low-resolution inputs, typically involve sophisticated architectures with substantial computational and memory demands, thereby limiting their practical applicability. The authors propose a novel framework, termed contrastive self-distillation (CSD), which is designed to compress and accelerate existing SISR models without compromising output quality.

The core of their method lies in the Channel-Splitting Super-Resolution network (CSSR-Net), which serves as a compact student network derived from a teacher network by utilizing only a portion of the teacher’s channels across all layers. This process inherently compresses the model, significantly reducing the number of parameters, while the network remains capable of dynamic inference depending on computational resource availability.

A salient feature of this work is the introduction of a novel contrastive loss which facilitates explicit knowledge transfer from the teacher network to CSSR-Net. This contrastive loss aims to optimize the image reconstruction quality by pulling the outputs of CSSR-Net closer to those of the teacher network while pushing them far from negative samples, such as bicubic upsampled images. This dual constraint helps in effectively reducing optimization space and enhancing the performance of the student network.

Extensive experiments validate the efficacy of the proposed CSD scheme across a range of standard SISR models like EDSR, RCAN, and CARN. The CSD framework demonstrates significant performance improvements in terms of both compression rates and speedup, making models suitable for deployment on devices with limited computational resources. For instance, the CSD compressed EDSR+ model achieves a 4× compression rate and 1.77× speedup while maintaining a minor loss of quality (0.13dB PSNR and 0.0039 SSIM) at a resolution scale of 4× on Urban100.

This paper opens new vistas for practical deployment of robust SR models on mobile and IoT devices, where computational capability is constrained. Beyond practical implications, the theoretical contribution lies in the integration of self-distillation within the domain of generative tasks such as super-resolution, with contrastive learning providing a novel avenue for achieving model compression without losing efficacy.

The approach employed in this paper suggests potential expansion into other edge-deployed AI applications like de-noising, de-hazing, or generative restoration tasks where resource efficiency is paramount. Future research could extend this method by exploring other forms of contrastive learning in conjunction with self-distillation frameworks, potentially giving rise to frameworks capable of overcoming even greater operational limitations while enhancing output quality.