Learning Convolutional Networks for Content-weighted Image Compression (1703.10553v2)

Abstract: Lossy image compression is generally formulated as a joint rate-distortion optimization to learn encoder, quantizer, and decoder. However, the quantizer is non-differentiable, and discrete entropy estimation usually is required for rate control. These make it very challenging to develop a convolutional network (CNN)-based image compression system. In this paper, motivated by that the local information content is spatially variant in an image, we suggest that the bit rate of the different parts of the image should be adapted to local content. And the content aware bit rate is allocated under the guidance of a content-weighted importance map. Thus, the sum of the importance map can serve as a continuous alternative of discrete entropy estimation to control compression rate. And binarizer is adopted to quantize the output of encoder due to the binarization scheme is also directly defined by the importance map. Furthermore, a proxy function is introduced for binary operation in backward propagation to make it differentiable. Therefore, the encoder, decoder, binarizer and importance map can be jointly optimized in an end-to-end manner by using a subset of the ImageNet database. In low bit rate image compression, experiments show that our system significantly outperforms JPEG and JPEG 2000 by structural similarity (SSIM) index, and can produce the much better visual result with sharp edges, rich textures, and fewer artifacts.

Content-weighted Image Compression Using Convolutional Neural Networks

The paper "Learning Convolutional Networks for Content-weighted Image Compression" authored by Mu Li et al. presents a convolutional neural network (CNN)-based framework for lossy image compression that aims to address fundamental challenges in conventional image compression methods such as JPEG and JPEG 2000. The primary focus of this research is on improving compression efficiency by considering the spatial variance in information content across different parts of an image and employing an end-to-end optimized approach.

Key Contributions

1. Content-weighted Importance Map: The paper introduces a content-weighted importance map that guides the allocation of bits based on local image content. This approach diverges from traditional compression standards that allocate bits uniformly, irrespective of spatial content significance. Instead, this method allows for spatially adaptive bit allocation, enhancing the retention of important structural features and textures in the compression process.
2. Differentiable Proxy Function for Binarization: A major challenge in CNN-based compression is the non-differentiability of the quantization step. The authors address this by employing a proxy function for the binary operation, allowing gradient-based optimization techniques to be applied effectively. This innovation enables the encoder, binarizer, decoder, and importance map to be jointly optimized.
3. End-to-End Optimization: The proposed system supports end-to-end training on image data, allowing all components to be optimized simultaneously using backpropagation. This holistic approach contrasts with traditional separate codec optimization, yielding improved compression outcomes, especially at low bit rates.
4. Experimental Superiority Over Conventional Methods: Empirical results demonstrate that the proposed method achieves superior structural similarity (SSIM) index scores compared to JPEG and JPEG 2000, especially evident at low bit rates. The visual quality of the output is also enhanced, maintaining sharper edges and higher quality textures, suggesting a better preservation of essential visual information.

Methodology and Architecture

The architecture developed in this paper comprises four components: a convolutional encoder, an importance map network, a binarizer, and a convolutional decoder. The encoder processes the input image to produce feature maps. The importance map is generated to determine the bit allocation strategy, which is quantized and modified based on importance levels, influencing the overall bit allocation efficacy.

To address the entropy coding process, which is disentangled from the rate-distortion optimization in this system, a convolutional entropy encoder is introduced for compressing the binary codes and the importance map. By training this with a smaller context than typically used in CABAC, the authors document further compression gains.

Implications and Future Prospects

This paper offers significant implications for the development of advanced image compression systems by leveraging deep learning methodologies, particularly for applications where preserving visual detail at low bit rates is crucial. Furthermore, the content-weighted approach offers a promising direction for adaptive data processing tasks where spatial information content variability is a key factor.

The integration of such systems into real-world applications could affect diverse domains like video streaming, medical imaging, and remote sensing, where image quality after compression can greatly impact user experience and analytical outcomes. Future research could explore augmenting the importance map with more complex feature assessments or integrating similar adaptive strategies in video compression and real-time communication systems. Moreover, there is potential for optimizing the computational efficiency of such models, ensuring they are suitable for use in resource-constrained environments.

In conclusion, this paper underscores the critical role of adaptive, content-aware strategies in image compression, leveraging the power of deep learning to markedly enhance the quality of compressed images while maintaining low bit rates.