LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyond (2105.10422v1)

Abstract: Single image super-resolution (SISR) deals with a fundamental problem of upsampling a low-resolution (LR) image to its high-resolution (HR) version. Last few years have witnessed impressive progress propelled by deep learning methods. However, one critical challenge faced by existing methods is to strike a sweet spot of deep model complexity and resulting SISR quality. This paper addresses this pain point by proposing a linearly-assembled pixel-adaptive regression network (LAPAR), which casts the direct LR to HR mapping learning into a linear coefficient regression task over a dictionary of multiple predefined filter bases. Such a parametric representation renders our model highly lightweight and easy to optimize while achieving state-of-the-art results on SISR benchmarks. Moreover, based on the same idea, LAPAR is extended to tackle other restoration tasks, e.g., image denoising and JPEG image deblocking, and again, yields strong performance. The code is available at https://github.com/dvlab-research/Simple-SR.

Overview of LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Image Super-Resolution and Beyond

The paper introduces LAPAR, a novel framework that addresses the critical challenge of balancing model complexity with the quality of single image super-resolution (SISR). Instead of directly mapping low-resolution (LR) to high-resolution (HR) images, LAPAR approaches this as a linear coefficient regression task using a dictionary of predefined filter bases. This method not only proves to be lightweight and efficient but also achieves state-of-the-art results across various SISR benchmarks. The authors further illustrate the versatility of LAPAR by applying it to other image restoration tasks, showing robust performance in image denoising and JPEG image deblocking.

Key Contributions

1. Efficient Regression Network: LAPAR reframes SISR as a regression problem where the task is to compute linear coefficients for pixel-adaptive filters that enhance bicubic interpolated images. The model's efficiency stems from this linear space constraint which simplifies optimization while ensuring robustness.
2. Predefined Filter Dictionaries: The strategy uses a dictionary of filters composed of Gaussian and difference of Gaussians (DoG) kernels. By fixing the filter bases and optimizing just the coefficients, the method circumvents the instability often seen in simultaneous optimization of filter dictionaries.
3. Lightweight Implementation and Performance: LAPAR demonstrates excellent results with fewer parameters compared to prevailing lightweight methods while maintaining low computational demands, as evidenced by the evaluation across Set5, Set14, B100, Urban100, and Manga109 datasets under multiple scale factors.
4. Extensibility to Other Tasks: Beyond SISR, LAPAR's architecture easily extends to tackle other restoration tasks, showcasing significant adaptability. Image denoising and JPEG deblocking experiments confirm its robust performance and general applicability.

Methodology and Results

The LAPAR framework leverages a lightweight convolutional neural network, termed $LaparNet$, which predicts spatially variant filter coefficients by analyzing the input image. These coefficients are then used in a linear combination with the predefined dictionary to reconstruct the HR image. The dictionary's filters are carefully chosen to maximize coverage of potential image attributes, supporting a wide variety of natural image complexities.

Empirical evaluations reveal LAPAR's strength, outperforming or matching state-of-the-art lightweight networks like CARN and FALSR on key metrics like PSNR and SSIM across standard benchmarks. Notably, LAPAR-A, the most robust configuration, distinctly leads in structural preservation and detail enhancement, particularly in complex scenes from the Urban100 dataset.

Implications and Future Directions

LAPAR’s formulation sets a promising direction for reducing the model size and computational burden in SISR tasks without sacrificing performance. This balancing act addresses scalability issues pertinent to deployment in resource-constrained environments such as mobile devices.

The paper's theoretical underpinnings suggest further exploration into more sophisticated filter dictionaries, possibly incorporating adaptive or learned filters to expand the model's generalization capabilities. Furthermore, integrating multi-task learning paradigms could enhance its application range, fostering advancements in the broader scope of image processing tasks.

In summary, LAPAR embodies an innovative pivot in SISR methodology, with compelling numerical outcomes underscoring its efficiency and precision. Its adaptable architecture signals potential for ongoing enhancements in the field of image restoration, advancing both practical applications and theoretical inquiry.