Latent Multi-Relation Reasoning for GAN-Prior based Image Super-Resolution (2208.02861v1)

Abstract: Recently, single image super-resolution (SR) under large scaling factors has witnessed impressive progress by introducing pre-trained generative adversarial networks (GANs) as priors. However, most GAN-Priors based SR methods are constrained by an attribute disentanglement problem in inverted latent codes which directly leads to mismatches of visual attributes in the generator layers and further degraded reconstruction. In addition, stochastic noises fed to the generator are employed for unconditional detail generation, which tends to produce unfaithful details that compromise the fidelity of the generated SR image. We design LAREN, a LAtent multi-Relation rEasoNing technique that achieves superb large-factor SR through graph-based multi-relation reasoning in latent space. LAREN consists of two innovative designs. The first is graph-based disentanglement that constructs a superior disentangled latent space via hierarchical multi-relation reasoning. The second is graph-based code generation that produces image-specific codes progressively via recursive relation reasoning which enables prior GANs to generate desirable image details. Extensive experiments show that LAREN achieves superior large-factor image SR and outperforms the state-of-the-art consistently across multiple benchmarks.

• The paper introduces LAREN, a novel technique that uses latent multi-relation reasoning to overcome GAN-prior challenges in high-factor image super-resolution.
• Graph-based disentanglement constructs a hierarchical latent space that preserves and accurately aligns visual attributes for improved reconstruction quality.
• Recursive code generation progressively produces image-specific codes that yield high-fidelity, detail-rich outputs in super-resolved images.

The paper, "Latent Multi-Relation Reasoning for GAN-Prior based Image Super-Resolution," addresses significant challenges in single image super-resolution (SR) where high scaling factors are required. The paper identifies two primary problems with existing GAN-prior based SR methods: the attribute disentanglement problem in inverted latent codes and the issue of stochastic noise leading to unfaithful detail generation.

To tackle these problems, the authors propose a novel technique named LAREN (LAtent multi-Relation rEasoNing). LAREN introduces two innovative designs to improve the performance of image SR:

1. Graph-Based Disentanglement: This approach constructs a more effective disentangled latent space via hierarchical multi-relation reasoning. The disentangled latent space helps in preserving and correctly aligning visual attributes, thus overcoming the mismatches in attribute representation that typically degrade the reconstruction quality.
2. Graph-Based Code Generation: This design progressively produces image-specific codes through recursive relation reasoning. The method leverages relationships within the latent space to generate codes that enable prior GANs to synthesize desirable and faithful image details. This step ensures that the generated high-resolution images maintain high fidelity to the original input.

By integrating these techniques, LAREN achieves significant improvements in large-factor image SR. Extensive experiments demonstrate that LAREN consistently outperforms state-of-the-art methods across multiple benchmarks. The results indicate that LAREN is highly effective in generating high-quality, high-resolution images, addressing both the attribute disentanglement issue and the unfaithful detail generation caused by traditional stochastic noise methods.