Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks (1809.00437v1)

Abstract: We consider the single image super-resolution problem in a more general case that the low-/high-resolution pairs and the down-sampling process are unavailable. Different from traditional super-resolution formulation, the low-resolution input is further degraded by noises and blurring. This complicated setting makes supervised learning and accurate kernel estimation impossible. To solve this problem, we resort to unsupervised learning without paired data, inspired by the recent successful image-to-image translation applications. With generative adversarial networks (GAN) as the basic component, we propose a Cycle-in-Cycle network structure to tackle the problem within three steps. First, the noisy and blurry input is mapped to a noise-free low-resolution space. Then the intermediate image is up-sampled with a pre-trained deep model. Finally, we fine-tune the two modules in an end-to-end manner to get the high-resolution output. Experiments on NTIRE2018 datasets demonstrate that the proposed unsupervised method achieves comparable results as the state-of-the-art supervised models.

• The paper presents a novel Cycle-in-Cycle GAN model that performs unsupervised super-resolution by coupling two CycleGANs.
• The methodology cleans noisy low-resolution images and then upscales them using a pre-trained EDSR network in an end-to-end adversarial framework.
• Experimental results on NTIRE2018 datasets show the unsupervised approach achieves competitive PSNR and SSIM performance compared to top supervised methods.

Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks

The paper under discussion addresses the challenge of single image super-resolution (SISR) under a more sophisticated scenario where low-/high-resolution image pairs and the down-sampling process are not accessible. This poses significant complexity, as traditional supervised deep learning methods cannot be directly applied due to their reliance on paired data and well-defined degradation processes. In this context, the authors propose an innovative approach utilizing unsupervised learning methods inspired by the principles of image-to-image translation within Generative Adversarial Networks (GANs).

The primary contribution of the paper is the development and evaluation of a novel Cycle-in-Cycle GAN (CinCGAN) model consisting of two coupled CycleGANs. The first CycleGAN is employed to map noisy and blurry low-resolution (LR) images to a noise-free low-resolution space. This ensures the cleaning of inputs, which is particularly vital since complex noises and blurring significantly hinder direct application of existing upscaling networks. Subsequent to this, a pre-trained deep model—specifically the Enhanced Deep Residual Network (EDSR)—is utilized to upscale these noiseless, intermediate LR images to the desired high-resolution (HR) outputs. The whole network is then fine-tuned in an end-to-end manner through bundled adversarial learning, making it robust against various unknown degradation processes.

The experimental results, conducted on the NTIRE2018 datasets, demonstrate the efficacy of this approach. Remarkably, the proposed unsupervised CinCGAN model achieves performance comparable to the leading supervised methods, a notable achievement given the lack of paired training data. The quantitative evaluations on critical metrics such as PSNR and SSIM show that the CinCGAN competes effectively against supervised networks fine-tuned on the target data and surpasses models relying on pre-processing like BM3D in combination with EDSR.

Dissecting these performances requires an appreciation of the architecture employed. The CinCGAN leverages a Cycle-in-Cycle structure where the second GAN encapsulates the first. This architectural design, coupled with strategic generator-discriminator setup and loss formulations, addresses two pivotal challenges simultaneously: denoising the LR inputs and efficiently mapping the clean intermediaries to high-quality HR images. This contrasts with simpler CycleGAN applications which struggle due to their incapacity to handle multiple transformations at once without paired data.

The work lays foundational insights for the development of adaptive and unsupervised SISR models, extending their feasibility in real-world scenarios where degradation processes are often complex and data pairing is unavailable. Future work can further explore variations in the architecture and loss formulations, potentially incorporating online-adaptive models that further refine the output based on immediate data characteristics or combinations with other advanced learning paradigms. The broader implications suggest a shift towards more flexible super-resolution systems that can cater to a diverse range of applications without restrictive training conditions. This aligns well with ongoing developments in deploying AI solutions where data privacy and acquisition pose significant constraints.