Optimizing Image Compression via Joint Learning with Denoising (2207.10869v1)

Abstract: High levels of noise usually exist in today's captured images due to the relatively small sensors equipped in the smartphone cameras, where the noise brings extra challenges to lossy image compression algorithms. Without the capacity to tell the difference between image details and noise, general image compression methods allocate additional bits to explicitly store the undesired image noise during compression and restore the unpleasant noisy image during decompression. Based on the observations, we optimize the image compression algorithm to be noise-aware as joint denoising and compression to resolve the bits misallocation problem. The key is to transform the original noisy images to noise-free bits by eliminating the undesired noise during compression, where the bits are later decompressed as clean images. Specifically, we propose a novel two-branch, weight-sharing architecture with plug-in feature denoisers to allow a simple and effective realization of the goal with little computational cost. Experimental results show that our method gains a significant improvement over the existing baseline methods on both the synthetic and real-world datasets. Our source code is available at https://github.com/felixcheng97/DenoiseCompression.

• The paper introduces a two-branch, weight-sharing architecture that jointly tackles image compression and denoising to optimize performance.
• It achieves superior rate-distortion results and significantly faster encoding times, particularly in high noise scenarios.
• Evaluation on synthetic and real-world (SIDD) datasets demonstrates enhanced restoration with preserved edge and texture details at lower bit rates.

Optimizing Image Compression via Joint Learning with Denoising

The paper addresses a notable challenge in the field of image compression: the detrimental effect of noise present in images captured by devices with small sensors, such as smartphones. Traditional lossy image compression techniques often misallocate resources by storing unnecessary noise alongside critical image details, resulting in reduced compression efficacy. The authors propose a novel method that optimizes image compression by jointly incorporating denoising capabilities into the compression process.

Methodology

The core of the paper introduces a two-branch, weight-sharing network architecture that separates denoising and compression tasks but enables joint learning. This approach incorporates efficient plug-in feature denoisers within the compression pipeline, aiming to remove noise preemptively from the compressed bitstreams while maintaining computational efficiency.

The network consists of an analysis transform, a synthesis transform, and a scale hyperprior model informed by the work of Ballé et al. (2018). The novelty lies in the integration of denoisers that clean the latent feature representations of noise. The training process involves two stages: a pre-training phase focusing on pure image compression and a fine-tuning phase where the network is exposed to noisy-clean image pairs to enhance denoising.

Evaluation

Extensive experiments were conducted on both synthetic and real-world datasets. In synthetic datasets, the proposed method showed superior rate-distortion (RD) performance compared to baseline methods, including sequential solutions combining traditional denoising and compression and a joint decompression-denoising baseline. This superiority was consistent across all levels of noise, notably at higher noise levels where it significantly outperformed alternatives.

For real-world scenarios, tested on the SIDD benchmark, the method also demonstrated high RD performance, indicating good generalization capability to real-world noise conditions.

Results and Implications

Key experimental results include an order of magnitude faster encoding times compared to sequential solutions, thanks to the computationally efficient design of the proposed network. Qualitatively, the method provides superior image restoration, with better preservation of edges and textures at lower bit rates than competing methods.

The joint compression and denoising methodology offer a practical solution for efficiently handling the noise in contemporary photographic devices without the drawbacks of a sequential denoising-compression pipeline. This work may inspire further research in joint processing pipelines for image and video compression, incorporating additional enhancements like super-resolution or image reconstruction under various noise and artifact conditions.

Future Work

Potential future avenues include extending the joint model to video compression, optimizing for other perceptually motivated metrics beyond PSNR and MS-SSIM, and exploring real-time application capabilities. The design could also be explored to support various types of noise beyond Gaussian, adapting to dynamic conditions of mobile photography. This paper paves the way for more advanced, noise-aware compression schemes that do not compromise image quality while optimizing storage and transmission resources.