Papers
Topics
Authors
Recent
Search
2000 character limit reached

NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results

Published 17 Apr 2024 in eess.IV and cs.AI | (2404.11313v1)

Abstract: This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i.e., Kuaishou/Kwai Platform. The KVQ database is divided into three parts, including 2926 videos for training, 420 videos for validation, and 854 videos for testing. The purpose is to build new benchmarks and advance the development of S-UGC VQA. The competition had 200 participants and 13 teams submitted valid solutions for the final testing phase. The proposed solutions achieved state-of-the-art performances for S-UGC VQA. The project can be found at https://github.com/lixinustc/KVQChallenge-CVPR-NTIRE2024.

Authors (68)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (84)
  1. Padding module: Learning the padding in deep neural networks. IEEE Access, 11:7348–7357, 2023.
  2. NTIRE 2024 dense and non-homogeneous dehazing challenge report. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  3. NTIRE 2024 challenge on night photography rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  4. Deep portrait quality assessment. a NTIRE 2024 challenge survey. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  5. Learning generalized spatial-temporal deep feature representation for no-reference video quality assessment. IEEE Trans. Circuits Syst. Video Technol., 32(4):1903–1916, 2022.
  6. NTIRE 2024 challenge on image super-resolution (×4): Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  7. No-reference blur assessment of digital pictures based on multifeature classifiers. IEEE Transactions on image processing, 20(1):64–75, 2010.
  8. Deep raw image super-resolution. a NTIRE 2024 challenge survey. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  9. Ugc-icvqa: Ugc video no-reference quality evaluation model based on complexity multi-task learning. 2024.
  10. Effects of padding on lstms and cnns. arXiv preprint arXiv:1903.07288, 2019.
  11. Perceptual quality assessment of smartphone photography. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3677–3686, 2020.
  12. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211, 2019.
  13. Massive online crowdsourced study of subjective and objective picture quality. IEEE Transactions on Image Processing, 25(1):372–387, 2015.
  14. Mahdi Hashemi. Enlarging smaller images before inputting into convolutional neural network: zero-padding vs. interpolation. Journal of Big Data, 6(1):1–13, 2019.
  15. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  16. The konstanz natural video database (konvid-1k). In QoMEX, pages 1–6. IEEE, 2017.
  17. Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Transactions on Image Processing, 29:4041–4056, 2020.
  18. Searching for mobilenetv3. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1314–1324, 2019.
  19. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size. ArXiv, abs/1602.07360, 2016.
  20. Frank Klinker. Exponential moving average versus moving exponential average. Mathematische Semesterberichte, 58:97–107, 2011.
  21. Ifrnet: Intermediate feature refine network for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1969–1978, 2022.
  22. Stablevqa: A deep no-reference quality assessment model for video stability. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1066–1076, 2023.
  23. Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of electronic imaging, 19(1):011006–011006, 2010.
  24. Blindly assess quality of in-the-wild videos via quality-aware pre-training and motion perception. IEEE Trans. Circuits Syst. Video Technol., 32(9):5944–5958, 2022.
  25. Quality assessment of in-the-wild videos. In ACM Multimedia, pages 2351–2359. ACM, 2019.
  26. Freqalign: Excavating perception-oriented transferability for blind image quality assessment from a frequency perspective. IEEE Transactions on Multimedia, 2023.
  27. NTIRE 2024 challenge on short-form UGC video quality assessment: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  28. NTIRE 2024 restore any image model (RAIM) in the wild challenge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  29. Kadid-10k: A large-scale artificially distorted iqa database. In 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), pages 1–3. IEEE, 2019.
  30. Partial convolution for padding, inpainting, and image synthesis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):6096–6110, 2022a.
  31. Ada-dqa: Adaptive diverse quality-aware feature acquisition for video quality assessment. CoRR, abs/2308.00729, 2023a.
  32. Ada-dqa: Adaptive diverse quality-aware feature acquisition for video quality assessment. In ACM Multimedia, pages 6695–6704. ACM, 2023b.
  33. Source-free unsupervised domain adaptation for blind image quality assessment. arXiv preprint arXiv:2207.08124, 2022b.
  34. Swiniqa: Learned swin distance for compressed image quality assessment. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 1795–1799, 2022c.
  35. Liqa: Lifelong blind image quality assessment. IEEE Transactions on Multimedia, 2022d.
  36. NTIRE 2024 quality assessment of AI-generated content challenge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024a.
  37. NTIRE 2024 challenge on low light image enhancement: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024b.
  38. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
  39. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12009–12019, 2022e.
  40. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11976–11986, 2022f.
  41. Video swin transformer. In CVPR, pages 3192–3201. IEEE, 2022g.
  42. Rtn: Reinforced transformer network for coronary ct angiography vessel-level image quality assessment. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 644–653. Springer, 2022a.
  43. Styleam: Perception-oriented unsupervised domain adaption for non-reference image quality assessment. arXiv preprint arXiv:2207.14489, 2022b.
  44. Aigc-vqa: A holistic perception metric for aigc video quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024a.
  45. Kvq: Kwai video quality assessment for short-form videos. CVPR, 2024b.
  46. Ava: A large-scale database for aesthetic visual analysis. In 2012 IEEE conference on computer vision and pattern recognition, pages 2408–2415. IEEE, 2012.
  47. Novel cnn with investigation on accuracy by modifying stride, padding, kernel size and filter numbers. Multimedia Tools and Applications, 82(15):23673–23691, 2023.
  48. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
  49. Codalab competitions: An open source platform to organize scientific challenges. Journal of Machine Learning Research, 24(198):1–6, 2023.
  50. Learning transferable visual models from natural language supervision. In ICML, pages 8748–8763. PMLR, 2021.
  51. The ninth NTIRE 2024 efficient super-resolution challenge report. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  52. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115:211 – 252, 2014.
  53. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on image processing, 15(11):3440–3451, 2006.
  54. A deep learning based no-reference quality assessment model for UGC videos. In ACM Multimedia, pages 856–865. ACM, 2022.
  55. Analysis of video quality datasets via design of minimalistic video quality models. arXiv preprint arXiv:2307.13981, 2023.
  56. Enhancing blind video quality assessment with rich quality-aware features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024.
  57. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural information processing systems, 35:10078–10093, 2022.
  58. NTIRE 2024 image shadow removal challenge report. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  59. Icme 2021 ugc-vqa challenge. In Available: http://ugcvqa.com/.
  60. NTIRE 2024 challenge on stereo image super-resolution: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024a.
  61. Youtube UGC dataset for video compression research. In MMSP, pages 1–5. IEEE, 2019.
  62. NTIRE 2024 challenge on light field image super-resolution: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024b.
  63. Convnext v2: Co-designing and scaling convnets with masked autoencoders. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16133–16142, 2023.
  64. Visual transformers: Where do transformers really belong in vision models? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 599–609, 2021.
  65. Fast-vqa: Efficient end-to-end video quality assessment with fragment sampling. In European conference on computer vision, pages 538–554. Springer, 2022a.
  66. Disentangling aesthetic and technical effects for video quality assessment of user generated content. CoRR, abs/2211.04894, 2022b.
  67. Discovqa: Temporal distortion-content transformers for video quality assessment. IEEE Trans. Circuits Syst. Video Technol., 33(9):4840–4854, 2023a.
  68. Towards explainable in-the-wild video quality assessment: A database and a language-prompted approach. In Proceedings of the 31st ACM International Conference on Multimedia, MM 2023, Ottawa, ON, Canada, 29 October 2023- 3 November 2023, pages 1045–1054. ACM, 2023b.
  69. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20144–20154, 2023c.
  70. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023d.
  71. Video quality assessment based on swin transformer with spatio-temporal feature fusion and data augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1846–1854, 2023e.
  72. Starvqa: Space-time attention for video quality assessment. In 2022 IEEE International Conference on Image Processing, ICIP 2022, Bordeaux, France, 16-19 October 2022, pages 2326–2330. IEEE, 2022.
  73. Short-form ugc video quality assessment based on multi-level video fusion with rank-aware. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024.
  74. NTIRE 2024 challenge on blind enhancement of compressed image: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  75. Patch-vq:’patching up’the video quality problem. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14019–14029, 2021.
  76. Sf-iqa: Quality and similarity integration for ai generated image quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024a.
  77. Video quality assessment based on swin transformerv2 and coarse to fine strategy. arXiv preprint arXiv:2401.08522, 2024b.
  78. Capturing co-existing distortions in user-generated content for no-reference video quality assessment. In ACM Multimedia, pages 1098–1107. ACM, 2023.
  79. NTIRE 2024 challenge on HR depth from images of specular and transparent surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  80. Sigmoid loss for language image pre-training. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11941–11952, 2023.
  81. Blind image quality assessment via vision-language correspondence: A multitask learning perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14071–14081, 2023.
  82. NTIRE 2024 challenge on bracketing image restoration and enhancement: Datasets, methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024.
  83. Quality-aware pretrained models for blind image quality assessment. In CVPR, pages 22302–22313. IEEE, 2023a.
  84. Zoom-vqa: Patches, frames and clips integration for video quality assessment. In CVPR Workshops, pages 1302–1310. IEEE, 2023b.
Citations (21)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up