Preparing VVC for Streaming: A Fast Multi-Rate Encoding Approach (2312.08330v1)
Abstract: The integration of advanced video codecs into the streaming pipeline is growing in response to the increasing demand for high quality video content. However, the significant computational demand for advanced codecs like Versatile Video Coding (VVC) poses challenges for service providers, including longer encoding time and higher encoding cost. This challenge becomes even more pronounced in streaming, as the same content needs to be encoded at multiple bitrates (also known as representations) to accommodate different network conditions. To accelerate the encoding process of multiple representations of the same content in VVC, we employ the encoding map of a single representation, known as the reference representation, and utilize its partitioning structure to accelerate the encoding of the remaining representations, referred to as dependent representations. To ensure compatibility with parallel processing, we designate the lowest bitrate representation as the reference representation. The experimental results indicate a substantial improvement in the encoding time for the dependent representations, achieving an average reduction of 40%, while maintaining a minimal average quality drop of only 0.43 in Video Multi-method Assessment Fusion (VMAF). This improvement is observed when utilizing Versatile Video Encoder (VVenC), an open and optimized VVC encoder implementation.
- “Overview of the H.264/AVC Video Coding Standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 560–576, July 2003.
- “Overview of the High Efficiency Video Coding (HEVC) Standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649–1668, Dec. 2012.
- “An Overview of Core Coding Tools in the AV1 Video Codec,” in 2018 Picture Coding Symposium (PCS), June 2018, pp. 41–45, ISSN: 2472-7822.
- Bitmovin, “6th Annual Video Developer Report,” 2022.
- “Overview of the Versatile Video Coding (VVC) Standard and its Applications,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 10, pp. 3736–3764, Oct. 2021.
- V Baroncini and M Wien, “VVC Verification Test Report for UHD SDR Video Content. Doc. JVET-T2020 of ITU-T/ISO/IEC Joint Video Experts Team (JVET),” in 20th JVET meeting: October, 2020.
- V Baroncini and M Wien, “Dry Run Subjective Assessment of SDR HD and 360 Video Verification Tests. Doc. JVET-U0119 of ITU-T/ISO/IEC Joint Video Experts Team (JVET),” in 21st JVET meeting: January, 2021.
- “VVC in the Cloud and Browser Playback: It Works,” in Proceedings of the 1st Mile-High Video Conference, Denver Colorado, Mar. 2022, pp. 19–24, ACM.
- “Vvenc: An Open And Optimized Vvc Encoder Implementation,” in 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Shenzhen, China, July 2021, pp. 1–2, IEEE.
- “Complexity Analysis Of Next-Generation VVC Encoding And Decoding,” in 2020 IEEE International Conference on Image Processing (ICIP), Oct. 2020, pp. 3134–3138, ISSN: 2381-8549.
- “CTU Depth Decision Algorithms for HEVC: A Survey,” Signal Processing: Image Communication, vol. 99, pp. 116442, Nov. 2021.
- “Light-Weight CNN-Based VVC Inter Partitioning Acceleration,” in 2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP), 2022, pp. 1–5.
- Y. Fan et al., “A Fast QTMT Partition Decision Strategy for VVC Intra Prediction,” IEEE Access, vol. 8, pp. 107900–107911, 2020.
- J. Cui et al., “Gradient-Based Early Termination of CU Partition in VVC Intra Coding,” in DCC, 2020, pp. 103–112.
- “Block Structure Reuse for Multi-rate High Efficiency Video Coding,” in 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, Sept. 2015, pp. 3972–3976, IEEE.
- “FaME-ML: Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning,” in 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), Macau, China, Dec. 2020, pp. 87–90, IEEE.
- “Fast Multi-Resolution and Multi-Rate Encoding for HTTP Adaptive Streaming Using Machine Learning,” IEEE Open Journal of Signal Processing, vol. 2, pp. 484–495, 2021.
- “Towards Optimal Multirate Encoding for HTTP Adaptive Streaming,” in MultiMedia Modeling, 2021, pp. 469–480.
- “EMES: Efficient Multi-Encoding Schemes for HEVC-based Adaptive Bitrate Streaming,” ACM Transactions on Multimedia Computing, Communications, and Applications, p. 3575659, Dec. 2022.
- “Block Partitioning Structure in the VVC Standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 10, pp. 3818–3833, Oct. 2021.
- “An Intra Subpartition Coding Mode for VVC,” in 2019 IEEE International Conference on Image Processing (ICIP), 2019, pp. 1203–1207.
- “Geometric Partitioning Mode in Versatile Video Coding: Algorithm Review and Analysis,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 9, pp. 3603–3617, 2021.
- “JVET-J1010: JVET Common Test Conditions and Software Reference Configurations,” 07 2018.
- “Statistical Analysis of Inter Coding in VVC Test Model (VTM),” in 2022 IEEE International Conference on Image Processing (ICIP), 2022, pp. 3456–3459.
- “AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling,” IEEE Transactions on Image Processing, vol. 32, pp. 251–266, 2023.
- “VCA: Video Complexity Analyzer,” in Proceedings of the 13th ACM Multimedia Systems Conference, June 2022, MMSys ’22, pp. 259–264.