3R-INN: How to be climate friendly while consuming/delivering videos?
Abstract: The consumption of a video requires a considerable amount of energy during the various stages of its life-cycle. With a billion hours of video consumed daily, this contributes significantly to the greenhouse gas emission. Therefore, reducing the end-to-end carbon footprint of the video chain, while preserving the quality of experience at the user side, is of high importance. To contribute in an impactful manner, we propose 3R-INN, a single light invertible network that does three tasks at once: given a high-resolution grainy image, it Rescales it to a lower resolution, Removes film grain and Reduces its power consumption when displayed. Providing such a minimum viable quality content contributes to reducing the energy consumption during encoding, transmission, decoding and display. 3R-INN also offers the possibility to restore either the high-resolution grainy original image or a grain-free version, thanks to its invertibility and the disentanglement of the high frequency, and without transmitting auxiliary data. Experiments show that, while enabling significant energy savings for encoding (78%), decoding (77%) and rendering (5% to 20%), 3R-INN outperforms state-of-the-art film grain synthesis and energy-aware methods and achieves state-of-the-art performance on the rescaling task on different test-sets.
- Vtm-19.0. https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/tags/VTM-19.0.
- Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 126–135, 2017.
- Style-based film grain analysis and synthesis. In Proceedings of the 14th Conference on ACM Multimedia Systems, pages 229–238, 2023.
- Deep-based film grain removal and synthesis. IEEE Transactions on Image Processing, 2023.
- Versatile video coding and super-resolution for efficient delivery of 8k video with 4k backward-compatibility. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2048–2052, 2020.
- Jvet-j1010: Jvet common test conditions and software reference configurations. In 10th Meeting of the Joint Video Experts Team, pages JVET–J1010, 2018.
- Invertible mosaic image hiding network for very large capacity image steganography. arXiv preprint arXiv:2309.08987, 2023.
- Film grain noise removal and synthesis in video coding. In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 890–893. IEEE, 2010.
- Display power modeling for energy consumption control. In 2023 IEEE International Conference on Image Processing (ICIP). IEEE, 2023.
- Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016.
- Hierarchical disentangled representation for invertible image denoising and beyond. arXiv preprint arXiv:2301.13358, 2023.
- Rich Franzen. Kodak lossless true color image suite. source: http://r0k. us/graphics/kodak, 4(2):9, 1999.
- Algorithm description for neural network-based video coding (nnvc-6.0). https://jvet-experts.org/doc_end_user/documents/31_Geneva/wg11/JVET-AE2019-v2.zip, 2023.
- Cristina Gomila. Sei message for film grain encoding. JVT document, May 2003, 2003.
- Sweet streams are made of this: The system engineer’s view on energy efficiency in video communications [feature]. IEEE Circuits and Systems Magazine, 23(1):57–77, 2023.
- Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
- Enhanced film grain noise removal for high fidelity video coding. In 2013 International Conference on Information Science and Cloud Computing Companion, pages 668–674. IEEE, 2013.
- Suk-Ju Kang. Image-quality-based power control technique for organic light emitting diode displays. Journal of Display Technology, 11(1):104–109, 2015.
- Image integrity-based gray-level error control for low power liquid crystal displays. IEEE Transactions on Consumer Electronics, 55(4):2401–2406, 2009.
- Task-aware image downscaling. In Proceedings of the European conference on computer vision (ECCV), pages 399–414, 2018.
- Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018.
- Invertible energy-aware images. IEEE Signal Processing Letters, 2023.
- Deep-learning-based energy aware images. In 2023 IEEE International Conference on Image Processing (ICIP), pages 590–594. IEEE, 2023.
- Invertible denoising network: A light solution for real noise removal. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13365–13374, 2021.
- Large-capacity image steganography based on invertible neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10816–10825, 2021.
- Jens Malmodin. The power consumption of mobile and fixed network data services-the case of streaming video and downloading large files. In Electronics Goes Green, volume 2020, 2020.
- A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, volume 2, pages 416–423. IEEE, 2001.
- A stochastic film grain model for resolution-independent rendering. In Computer Graphics Forum, volume 36, pages 684–699. Wiley Online Library, 2017.
- Film grain synthesis for av1 video codec. In 2018 Data Compression Conference, pages 3–12. IEEE, 2018.
- R-ace network for oled image power saving. In 2022 IEEE 4th Global Conference on Life Sciences and Technologies (LifeTech), pages 284–285. IEEE, 2022.
- Implementation of film-grain technology within vvc. In Applications of Digital Image Processing XLIV, volume 11842, pages 85–95. SPIE, 2021.
- On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237, 2019.
- Pixel value adjustment to reduce the energy requirements of display devices. SMPTE Motion Imaging Journal, 132(7):10–19, 2023.
- Dom Robinson. Greening of streaming: The less accord: Low energy sustainable streaming. In Proceedings of the 2nd Mile-High Video Conference (MHV’23), page 115, 2023.
- Unsupervised deep power saving and contrast enhancement for oled displays. arXiv preprint arXiv:1905.05916, 2019.
- Stochastic geometry and its applications. John Wiley & Sons, 2013.
- Learned image downscaling for upscaling using content adaptive resampler. IEEE Transactions on Image Processing, 29:4027–4040, 2020.
- The Carbon Trust. Carbon impact of video streaming. https://www.carbontrust.com/en-eu/node/1537, 2021.
- Invertible image rescaling. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 126–144. Springer, 2020.
- Deep battery saver: End-to-end learning for power constrained contrast enhancement. IEEE Transactions on Multimedia, 23:1049–1059, 2020.
- Invertible image decolorization. IEEE Transactions on Image Processing, 30:6081–6095, 2021.
- Blind image denoising via dependent dirichlet process tree. IEEE transactions on pattern analysis and machine intelligence, 39(8):1518–1531, 2016.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.