Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 77 tok/s
Gemini 2.5 Pro 45 tok/s Pro
GPT-5 Medium 30 tok/s Pro
GPT-5 High 33 tok/s Pro
GPT-4o 101 tok/s Pro
Kimi K2 168 tok/s Pro
GPT OSS 120B 452 tok/s Pro
Claude Sonnet 4.5 36 tok/s Pro
2000 character limit reached

Towards Backward-Compatible Continual Learning of Image Compression (2402.18862v1)

Published 29 Feb 2024 in eess.IV

Abstract: This paper explores the possibility of extending the capability of pre-trained neural image compressors (e.g., adapting to new data or target bitrates) without breaking backward compatibility, the ability to decode bitstreams encoded by the original model. We refer to this problem as continual learning of image compression. Our initial findings show that baseline solutions, such as end-to-end fine-tuning, do not preserve the desired backward compatibility. To tackle this, we propose a knowledge replay training strategy that effectively addresses this issue. We also design a new model architecture that enables more effective continual learning than existing baselines. Experiments are conducted for two scenarios: data-incremental learning and rate-incremental learning. The main conclusion of this paper is that neural image compressors can be fine-tuned to achieve better performance (compared to their pre-trained version) on new data and rates without compromising backward compatibility. Our code is available at https://gitlab.com/viper-purdue/continual-compression

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. The jpeg ai standard: Providing efficient human and machine visual data consumption. IEEE MultiMedia, 30(1):100–111, 2023.
  2. Integer networks for data compression with latent-variable models. International Conference on Learning Representations, 2018.
  3. Variational image compression with a scale hyperprior. International Conference on Learning Representations, 2018.
  4. Nonlinear transform coding. IEEE Journal of Selected Topics in Signal Processing, 15(2):339–353, 2021.
  5. Gisle Bjontegaard. Calculation of average psnr differences between rd-curves. Video Coding Experts Group - M33, 2001.
  6. High-fidelity variable-rate image compression via invertible activation transformation. Proceedings of the ACM International Conference on Multimedia, pages 2021–2031, 2022.
  7. Content adaptive optimization for neural image compression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
  8. Modeling the background for incremental learning in semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9233–9242, 2020.
  9. Variable bitrate image compression with quality scaling factors. IEEE International Conference on Acoustics, Speech and Signal Processing, pages 2163–2167, 2020.
  10. End-to-end learnt image compression via non-local attention optimization and improved context modeling. IEEE Transactions on Image Processing, 30:3179–3191, 2021.
  11. Learned image compression with discretized gaussian mixture likelihoods and attention modules. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7936–7945, 2020.
  12. Variable rate deep image compression with a conditional autoencoder. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3146–3154, 2019.
  13. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3366–3385, 2022.
  14. Qarv: Quantization-aware resnet vae for lossy image compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–15, 2023a.
  15. Lossy image compression with quantized hierarchical vaes. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 198–207, 2023b.
  16. The use of asymmetric numeral systems as an accurate replacement for huffman coding. Picture Coding Symposium, pages 65–69, 2015.
  17. Image compression with product quantized masked image modeling. Transactions on Machine Learning Research, 2023.
  18. Nvtc: Nonlinear vector transform coding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6101–6110, 2023.
  19. Flexible neural image compression via code editing. Advances in Neural Information Processing Systems, 35:12184–12196, 2022.
  20. Neural image compression via attentional multi-scale back projection and frequency decomposition. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14677–14686, 2021.
  21. Soft then hard: Rethinking the quantization in neural image compression. Proceedings of the International Conference on Machine Learning, 139:3920–3929, 2021.
  22. Checkerboard context model for efficient learned image compression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14766–14775, 2021.
  23. Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5708–5717, 2022.
  24. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  25. Learning end-to-end lossy image compression: A benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4194–4211, 2022.
  26. MLIC$^{++}$: Linear complexity multi-reference entropy modeling for learned image compression. ICML Workshop Neural Compression: From Information Theory to Applications, 2023.
  27. Mlic: Multi-reference entropy model for learned image compression. Proceedings of the ACM International Conference on Multimedia, pages 7618–7627, 2023.
  28. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017.
  29. Device interoperability for learned image compression with weights and activations quantization. Picture Coding Symposium, pages 151–155, 2022.
  30. Maskgan: Towards diverse and interactive facial image manipulation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5548–5557, 2020.
  31. Selective compression learning of latent representations for variable-rate image compression. Advances in Neural Information Processing Systems, 35:13146–13157, 2022.
  32. Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, pages 740–755, 2014.
  33. Learned image compression with mixed transformer-cnn architectures. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14388–14397, 2023.
  34. Mnemonics training: Multi-class incremental learning without forgetting. Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 12245–12254, 2020.
  35. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11966–11976, 2022.
  36. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
  37. High-efficiency lossy image coding through adaptive neighborhood information aggregation. arXiv preprint arXiv:2204.11448, 2022.
  38. Transformer-based image compression. Data Compression Conference, pages 469–469, 2022.
  39. End-to-end optimized versatile image compression with wavelet-like transform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3):1247–1263, 2022.
  40. Channel-wise autoregressive entropy models for learned image compression. Proceedings of the IEEE International Conference on Image Processing, pages 3339–3343, 2020.
  41. Joint autoregressive and hierarchical priors for learned image compression. Advances in Neural Information Processing Systems, 31:10794–10803, 2018.
  42. Content adaptive latents and decoder for neural image compression. Proceedings of the European Conference on Computer Vision, pages 556–573, 2022.
  43. Learning accurate entropy model with global reference for image compression. International Conference on Learning Representations, 2021.
  44. Entroformer: A transformer-based entropy model for learned image compression. International Conference on Learning Representations, 2022.
  45. icarl: Incremental classifier and representation learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5533–5542, 2017.
  46. Arithmetic coding. IBM Journal of Research and Development, 23(2):149–162, 1979.
  47. Dec-adapter: Exploring efficient decoder-side adapter for bridging screen content and natural image compression. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12887–12896, 2023.
  48. Variable-rate deep image compression through spatially-adaptive feature transform. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2360–2369, 2021.
  49. Effortless cross-platform video codec: A codebook-based method. arXiv preprint arXiv:2310.10292, 2023.
  50. Universal deep image compression via content-adaptive optimization with adapters. IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2528–2537, 2023.
  51. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00487, 2023.
  52. Enhanced invertible encoding for learned image compression. Proceedings of the ACM International Conference on Multimedia, pages 162–170, 2021.
  53. Slimmable compressive autoencoders for practical neural image compression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4996–5005, 2021.
  54. Computationally-efficient neural image compression with shallow decoders. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 530–540, 2023.
  55. Variational Bayesian quantization. Proceedings of the International Conference on Machine Learning, 119:10670–10680, 2020a.
  56. Improving inference for neural image compression. Advances in Neural Information Processing Systems, 33:573–584, 2020b.
  57. Lvqac: Lattice vector quantization coupled with spatially adaptive companding for efficient learned image compression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10239–10248, 2023.
  58. Unified multivariate gaussian mixture for efficient neural image compression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17591–17600, 2022.
  59. The devil is in the details: Window-based attention for image compression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17471–17480, 2022.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up