Lossy Image Compression with Compressive Autoencoders (1703.00395v1)

Abstract: We propose a new approach to the problem of optimizing autoencoders for lossy image compression. New media formats, changing hardware technology, as well as diverse requirements and content types create a need for compression algorithms which are more flexible than existing codecs. Autoencoders have the potential to address this need, but are difficult to optimize directly due to the inherent non-differentiabilty of the compression loss. We here show that minimal changes to the loss are sufficient to train deep autoencoders competitive with JPEG 2000 and outperforming recently proposed approaches based on RNNs. Our network is furthermore computationally efficient thanks to a sub-pixel architecture, which makes it suitable for high-resolution images. This is in contrast to previous work on autoencoders for compression using coarser approximations, shallower architectures, computationally expensive methods, or focusing on small images.

Lossy Image Compression with Compressive Autoencoders

The paper "Lossy Image Compression with Compressive Autoencoders" by Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszár introduces a novel approach to optimizing autoencoders for the task of lossy image compression. This method demonstrates competitive performance with JPEG 2000 and surpasses recently proposed techniques based on Recurrent Neural Networks (RNNs). The proposed network architecture leverages a sub-pixel structure to enhance computational efficiency, thus making it viable for high-resolution images without the drawbacks of prior approaches focusing on small images or computationally intensive methods.

The method pivots on addressing the non-differentiability inherent in the lossy compression process, particularly the quantization phase, which obstructs the direct use of gradient-based optimization techniques. The researchers introduce a surrogate gradient approach to circumvent this issue, allowing the training of deep autoencoders. This surrogate gradient approximates the rounding function's derivative during the backward pass, ensuring the quantization is preserved during the forward pass.

Compressive Autoencoder Architecture

The compressive autoencoder (CAE) comprises an encoder, a decoder, and a probabilistic model used for entropy coding. The loss function optimizes a trade-off between the number of bits used and the distortion introduced, defined as: $$-\log_2 Q\left( \left[f(\mathbf{x})\right] \right) + \beta \cdot d\left( \mathbf{x}, g( \left[f(\mathbf{x})\right] ) \right)$$ where $\beta$ controls the trade-off between bit rate and distortion.

Handling Non-Differentiability

Quantization, being a non-differentiable operation, presents a significant hurdle in training these networks. The authors propose various techniques, including:

1. Rounding Approximation: Replace the rounding function's derivative with a smooth approximation during backpropagation, which effectively allows gradients to propagate without altering the forward pass operation.
2. Stochastic Methods: Consider alternatives like stochastic rounding and additive noise, acknowledging their impact on the error signals received by the autoencoder during the training phase.

Entropy Rate Estimation

To address the non-differentiable nature of the entropy rate computation, the authors employ a continuous approximation for the discrete distribution $Q$ by leveraging a probability density function $q$ and ensuring differentiability through Jensen's inequality. This provides an upper bound for the entropy rate, facilitating gradient computation for training.

Experimental Evaluation

The performance of the proposed CAE is validated against JPEG, JPEG 2000, and prior RNN-based approaches. Evaluations consider metrics such as PSNR, SSIM, and MS-SSIM over the Kodak Image Dataset. The CAE method outperforms in terms of SSIM scores, achieving visually superior results, particularly at higher bit rates. Mean opinion score (MOS) tests further substantiate the perceptual quality advantages of CAE over traditional methods like JPEG and JPEG 2000, demonstrating the efficacy of the proposed approach.

Implications and Future Directions

This work implies significant advancements both theoretically and practically in the field of adaptive image compression. By facilitating end-to-end optimization using neural networks, the CAE approach can be tailored to various content types and quality metrics more readily than traditional codecs.

Future research directions as highlighted by the authors include optimizing the CAE for different perceptually relevant metrics and investigating the integration of generative models like GANs for further enhancement of perceptual quality. Additionally, the impending ubiquity of hardware accelerated neural networks could catalyze practical deployments of CAE in diverse applications from mobile devices to VR content.

In conclusion, this paper showcases a promising paradigm in image compression, leveraging the capabilities of deep learning to surpass conventional methods in flexibility and performance. The proposed methodologies for dealing with non-differentiability and efficient training strategies underscore a pivotal stride towards more adaptable and high-performance image compression algorithms.