Conditional Hallucinations for Image Compression
Abstract: In lossy image compression, models face the challenge of either hallucinating details or generating out-of-distribution samples due to the information bottleneck. This implies that at times, introducing hallucinations is necessary to generate in-distribution samples. The optimal level of hallucination varies depending on image content, as humans are sensitive to small changes that alter the semantic meaning. We propose a novel compression method that dynamically balances the degree of hallucination based on content. We collect data and train a model to predict user preferences on hallucinations. By using this prediction to adjust the perceptual weight in the reconstruction loss, we develop a Conditionally Hallucinating compression model (ConHa) that outperforms state-of-the-art image compression methods. Code and images are available at https://polybox.ethz.ch/index.php/s/owS1k5JYs4KD4TA.
- High-fidelity generative image compression. Advances in Neural Information Processing Systems, 33:11913–11924, 2020.
- Po-elic: Perception-oriented efficient learned image coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1764–1769, 2022.
- Gregory K Wallace. The jpeg still picture compression standard. Communications of the ACM, 34(4):30–44, 1991.
- Jpeg2000: Standard for interactive imaging. Proceedings of the IEEE, 90(8):1336–1357, 2002.
- Google Developers. Webp. https://developers.google.com/speed/webp/. Accessed: May 17, 2024.
- Fabrice Bellard. BPG (Better Portable Graphics). https://bellard.org/bpg/. Accessed: May 17, 2024.
- Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
- Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning, pages 1278–1286. PMLR, 2014.
- End-to-end optimized image compression. arXiv preprint arXiv:1611.01704, 2016.
- Variational image compression with a scale hyperprior. arXiv preprint arXiv:1802.01436, 2018.
- Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5718–5727, 2022.
- Compression.cc. Compression.cc. https://www.compression.cc/. Accessed on May 17, 2024.
- The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018.
- Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
- Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
- Real-time adaptive image compression. In International Conference on Machine Learning, pages 2922–2930. PMLR, 2017.
- Generative compression. In 2018 Picture Coding Symposium (PCS), pages 258–262. IEEE, 2018.
- Deep generative models for distribution-preserving lossy compression. Advances in neural information processing systems, 31, 2018.
- Generative adversarial networks for extreme learned image compression. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 221–231, 2019.
- Rethinking lossy compression: The rate-distortion-perception tradeoff. In International Conference on Machine Learning, pages 675–685. PMLR, 2019.
- Universal rate-distortion-perception representations for lossy compression. Advances in Neural Information Processing Systems, 34:11517–11529, 2021.
- Multi-realism image compression with a conditional generator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22324–22333, 2023.
- Controlling rate, distortion, and realism: Towards a single comprehensive neural image compression model. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2900–2909, 2024.
- Div2k dataset, 2017. Accessed: 2024-05-22.
- TOFlow: Temporal output learning for video frame interpolation. http://toflow.csail.mit.edu/, 2019.
- Kodak graphics. http://r0k.us/graphics/kodak/.
- Mark E Glickman. A comprehensive guide to chess ratings. American Chess Journal, 3(1):59–102, 1995.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.