Papers
Topics
Authors
Recent
2000 character limit reached

THQA: A Perceptual Quality Assessment Database for Talking Heads

Published 13 Apr 2024 in cs.CV and eess.IV | (2404.09003v1)

Abstract: In the realm of media technology, digital humans have gained prominence due to rapid advancements in computer technology. However, the manual modeling and control required for the majority of digital humans pose significant obstacles to efficient development. The speech-driven methods offer a novel avenue for manipulating the mouth shape and expressions of digital humans. Despite the proliferation of driving methods, the quality of many generated talking head (TH) videos remains a concern, impacting user visual experiences. To tackle this issue, this paper introduces the Talking Head Quality Assessment (THQA) database, featuring 800 TH videos generated through 8 diverse speech-driven methods. Extensive experiments affirm the THQA database's richness in character and speech features. Subsequent subjective quality assessment experiments analyze correlations between scoring results and speech-driven methods, ages, and genders. In addition, experimental results show that mainstream image and video quality assessment methods have limitations for the THQA database, underscoring the imperative for further research to enhance TH video quality assessment. The THQA database is publicly accessible at https://github.com/zyj-2000/THQA.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. “Perceptual quality assessment for digital human heads,” in ICASSP. IEEE, 2023, pp. 1–5.
  2. “Ddh-qa: A dynamic digital humans quality assessment database,” in ICME. IEEE, 2023, pp. 2519–2524.
  3. “Advancing zero-shot digital human quality assessment through text-prompted evaluation,” arXiv preprint arXiv:2307.02808, 2023.
  4. “Quality-of-experience evaluation for digital twins in 6g network environments,” IEEE Transactions on Broadcasting, 2024.
  5. “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems, vol. 30, 2017.
  6. “A no-reference image blur metric based on the cumulative probability of blur detection (cpbd),” IEEE TIP, vol. 20, no. 9, pp. 2678–2683, 2011.
  7. “Subjective and objective quality assessment for in-the-wild computer graphics images,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 20, no. 4, pp. 1–22, 2023.
  8. “An implementation of multimodal fusion system for intelligent digital human generation,” arXiv preprint arXiv:2310.20251, 2023.
  9. “A style-based generator architecture for generative adversarial networks,” in IEEE/CVF CVPR, 2019, pp. 4401–4410.
  10. “Synthesizing obama: learning lip sync from audio,” ACM ToG, vol. 36, no. 4, pp. 1–13, 2017.
  11. “Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,” in IEEE/CVF CVPR, 2023, pp. 8652–8661.
  12. “Makelttalk: speaker-aware talking-head animation,” ACM TOG, vol. 39, no. 6, pp. 1–15, 2020.
  13. “Audio2head: Audio-driven one-shot talking-head generation with natural head motion,” arXiv preprint arXiv:2107.09293, 2021.
  14. “Identity-preserving talking face generation with landmark and appearance priors,” in IEEE/CVF CVPR, June 2023, pp. 9729–9738.
  15. “Dreamtalk: When expressive talking head generation meets diffusion probabilistic models,” arXiv preprint arXiv:2312.09767, 2023.
  16. “A lip sync expert is all you need for speech to lip generation in the wild,” in Proceedings of the 28th ACM international conference on multimedia, 2020, pp. 484–492.
  17. “Videoretalking: Audio-based lip synchronization for talking head video editing in the wild,” in SIGGRAPH Asia 2022, 2022, pp. 1–9.
  18. “Dinet: Deformation inpainting network for realistic face visually dubbing on high resolution video,” arXiv preprint arXiv:2303.03988, 2023.
  19. “A no-reference quality assessment method for digital human head,” in ICIP. IEEE, 2023, pp. 36–40.
  20. “Geometry-aware video quality assessment for dynamic digital human,” in ICIP. IEEE, 2023, pp. 1365–1369.
  21. “A no-reference quality assessment metric for dynamic 3d digital human,” Displays, vol. 80, pp. 102540, 2023.
  22. “A reduced-reference quality assessment metric for textured mesh digital humans,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 2965–2969.
  23. “Stablevqa: A deep no-reference quality assessment model for video stability,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 1066–1076.
  24. “Light-vqa: A multi-dimensional quality assessment model for low-light video enhancement,” in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 1088–1097.
  25. “Towards open-ended visual quality comparison,” arXiv preprint arXiv:2402.16641, 2024.
  26. RECOMMENDATION ITU-R BT, “Methodology for the subjective assessment of the quality of television pictures,” International Telecommunication Union, 2002.
  27. “Agiqa-3k: An open database for ai-generated image quality assessment,” IEEE Transactions on Circuits and Systems for Video Technology, 2023.
  28. “No-reference image quality assessment in the spatial domain,” IEEE TIP, vol. 21, no. 12, pp. 4695–4708, 2012.
  29. “Making a “completely blind” image quality analyzer,” IEEE SPL, vol. 20, no. 3, pp. 209–212, 2012.
  30. “A feature-enriched completely blind image quality evaluator,” IEEE TIP, vol. 24, no. 8, pp. 2579–2591, 2015.
  31. “A completely blind video integrity oracle,” IEEE TIP, vol. 25, no. 1, pp. 289–300, 2015.
  32. “Blind prediction of natural video quality,” IEEE TIP, vol. 23, no. 3, pp. 1352–1365, 2014.
  33. Jari Korhonen, “Two-level approach for no-reference consumer video quality assessment,” IEEE TIP, vol. 28, no. 12, pp. 5923–5938, 2019.
  34. “Ugc-vqa: Benchmarking blind video quality assessment for user generated content,” IEEE TIP, vol. 30, pp. 4449–4464, 2021.
  35. “Quality assessment of in-the-wild videos,” in ACM MM, 2019, pp. 2351–2359.
  36. “Rapique: Rapid and accurate video quality prediction of user generated content,” IEEE DOAJ, vol. 2, pp. 425–440, 2021.
  37. “A deep learning based no-reference quality assessment model for ugc videos,” in ACM MM, 2022.
  38. “Fast-vqa: Efficient end-to-end video quality assessment with fragment sampling,” in ECCV. 2022, pp. 538–554, Springer.
  39. “Blindly assess quality of in-the-wild videos via quality-aware pre-training and motion perception,” IEEE TCSVT, vol. 32, no. 9, pp. 5944–5958, 2022.
Citations (9)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up