Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression (2203.10897v1)

Abstract: Modeling latent variables with priors and hyperpriors is an essential problem in variational image compression. Formally, trade-off between rate and distortion is handled well if priors and hyperpriors precisely describe latent variables. Current practices only adopt univariate priors and process each variable individually. However, we find inter-correlations and intra-correlations exist when observing latent variables in a vectorized perspective. These findings reveal visual redundancies to improve rate-distortion performance and parallel processing ability to speed up compression. This encourages us to propose a novel vectorized prior. Specifically, a multivariate Gaussian mixture is proposed with means and covariances to be estimated. Then, a novel probabilistic vector quantization is utilized to effectively approximate means, and remaining covariances are further induced to a unified mixture and solved by cascaded estimation without context models involved. Furthermore, codebooks involved in quantization are extended to multi-codebooks for complexity reduction, which formulates an efficient compression procedure. Extensive experiments on benchmark datasets against state-of-the-art indicate our model has better rate-distortion performance and an impressive $3.18\times$ compression speed up, giving us the ability to perform real-time, high-quality variational image compression in practice. Our source code is publicly available at \url{https://github.com/xiaosu-zhu/McQuic}.

• The paper introduces a multivariate Gaussian mixture model that captures inter- and intra-correlations in latent variables to improve rate-distortion performance.
• It presents a probabilistic vector quantization and cascaded estimation scheme that reduces model complexity while boosting efficiency in neural image compression.
• Empirical results demonstrate a 3.18× speedup in compression latency and superior rate-distortion metrics compared to state-of-the-art methods.

Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression

The paper presents a novel approach to neural image compression by introducing a unified multivariate Gaussian mixture model to efficiently handle the rate-distortion trade-off in image compression. Traditional models typically utilize univariate priors, processing variables individually. This research recognizes visual redundancies in latent variables when observed from a vectorized perspective, finding both inter-correlations and intra-correlations. These insights enhance rate-distortion performance and improve parallel processing capacity, accelerating the compression process.

Key Methodological Contributions

1. Vectorized Prior in Latent Variables: The paper proposes a multivariate Gaussian mixture model for latent variables, diverging from conventional univariate models. This model accounts for inter- and intra-correlations through means and covariances, providing a more comprehensive statistical description of latents derived from image data.
2. Probabilistic Vector Quantization: A novel probabilistic vector quantization method is introduced to approximate these means and progressively estimate covariances. As opposed to traditional methods that employ deterministic nearest neighbor approaches, this method leverages stochastic sampling to estimate the parameters, potentially enhancing the robustness of the training process.
3. Cascaded Estimation Scheme: The model employs a cascaded estimation framework, iteratively quantizing and estimating latent variables without relying on context models, which are common in conventional approaches. This reduces model complexity and increases computational efficiency.
4. Introduction of Multi-Codebooks: To handle compression complexity and allow flexible rate control, the paper extends its quantization approach by introducing multiple codebooks. This strategy enables the efficient management of larger vector spaces without a proportional increase in computational burden.

Empirical Evaluation

The experimental results demonstrate that the proposed model achieves superior rate-distortion performance compared to state-of-the-art methods across benchmark datasets. Notably, the model achieves an impressive $3.18\times$ speedup in compression latency, a significant enhancement for real-time applications. The research also supports these findings with extensive ablation studies, confirming the benefit of vectorized priors and the multivariate Gaussian approach.

Implications and Future Directions

From a practical standpoint, this approach suggests a pathway to more efficient and effective neural image compression systems that can operate in real-time without sacrificing performance. The integration of multivariate approaches could be further explored in other domains of data compression and representation learning, suggesting broader applicability beyond image processing.

This research drives future work towards improved compression models with robust rate control mechanisms. Further advancements could explore adaptive mechanisms that allow dynamic changes to the number of codebooks during compression, or mechanisms to automate the tuning of model parameters in response to different types of image data or computational constraints.

In summary, the paper provides a significant extension to conventional neural compression methodologies, combining theoretical insights with practical enhancements to achieve better computational efficiency and compression efficacy.