Papers
Topics
Authors
Recent
Search
2000 character limit reached

Super Efficient Neural Network for Compression Artifacts Reduction and Super Resolution

Published 26 Jan 2024 in cs.CV and eess.IV | (2401.14641v1)

Abstract: Video quality can suffer from limited internet speed while being streamed by users. Compression artifacts start to appear when the bitrate decreases to match the available bandwidth. Existing algorithms either focus on removing the compression artifacts at the same video resolution, or on upscaling the video resolution but not removing the artifacts. Super resolution-only approaches will amplify the artifacts along with the details by default. We propose a lightweight convolutional neural network (CNN)-based algorithm which simultaneously performs artifacts reduction and super resolution (ARSR) by enhancing the feature extraction layers and designing a custom training dataset. The output of this neural network is evaluated for test streams compressed at low bitrates using variable bitrate (VBR) encoding. The output video quality shows a 4-6 increase in video multi-method assessment fusion (VMAF) score compared to traditional interpolation upscaling approaches such as Lanczos or Bicubic.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (22)
  1. Fast, accurate, and lightweight super-resolution with cascading residual network. In Computer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part X, page 256–272, Berlin, Heidelberg, 2018. Springer-Verlag.
  2. Supervegan: Super resolution video enhancement gan for perceptually improving low bitrate streams. In IEEE Access, volume 9, pages 91160–91174, 2021.
  3. Collapsible linear blocks for super-efficient super resolution. In D. Marculescu, Y. Chi, and C. Wu, editors, Proceedings of Machine Learning and Systems, volume 4, pages 529–547, 2022.
  4. Basicvsr++: Improving video super-resolution with enhanced propagation and alignment. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5962–5971, 2022.
  5. Neural network-based video compression artifact reduction using temporal correlation and sparsity prior predictions. IEEE Access, 8:162479–162490, 2020.
  6. Compression artifacts reduction by a deep convolutional network. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 576–584, 2015.
  7. Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell., 38(2):295–307, feb 2016.
  8. Accelerating the super-resolution convolutional neural network. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision – ECCV 2016, pages 391–407, Cham, 2016. Springer International Publishing.
  9. Anchor-based plain net for mobile image super-resolution. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2494–2502, Los Alamitos, CA, USA, jun 2021. IEEE Computer Society.
  10. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
  11. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2704–2713, 2018.
  12. Video super-resolution with convolutional neural networks. IEEE transactions on computational imaging, 2(2):109–122, 2016.
  13. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012.
  14. Deep laplacian pyramid networks for fast and accurate super-resolution. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5835–5843, 2017.
  15. Toward a practical perceptual video quality metric. The Netflix Tech Blog, 6(2):2, 2016.
  16. Deep perceptual image quality assessment for compression. In 2021 IEEE International Conference on Image Processing (ICIP), pages 1484–1488. IEEE, 2021.
  17. Power-of-two quantization for low bitwidth and hardware compliant neural networks. arXiv preprint arXiv:2203.05025, 2022.
  18. Reduction of video compression artifacts based on deep temporal networks. IEEE Access, 6:63094–63106, 2018.
  19. Adapool: Exponential adaptive pooling for information-retaining downsampling. IEEE Transactions on Image Processing, 32:251–266, 2022.
  20. Ntire 2018 challenge on single image super-resolution: Methods and results. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 852–863, 2018.
  21. Video enhancement with task-oriented flow. International Journal of Computer Vision, 127:1106–1125, 2019.
  22. Ntire 2022 challenge on super-resolution and quality enhancement of compressed video: Dataset, methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1221–1238, 2022.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up