Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessment (2404.03407v1)

Published 4 Apr 2024 in cs.CV

Abstract: With the rapid advancements in AI-Generated Content (AIGC), AI-Generated Images (AIGIs) have been widely applied in entertainment, education, and social media. However, due to the significant variance in quality among different AIGIs, there is an urgent need for models that consistently match human subjective ratings. To address this issue, we organized a challenge towards AIGC quality assessment on NTIRE 2024 that extensively considers 15 popular generative models, utilizing dynamic hyper-parameters (including classifier-free guidance, iteration epochs, and output image resolution), and gather subjective scores that consider perceptual quality and text-to-image alignment altogether comprehensively involving 21 subjects. This approach culminates in the creation of the largest fine-grained AIGI subjective quality database to date with 20,000 AIGIs and 420,000 subjective ratings, known as AIGIQA-20K. Furthermore, we conduct benchmark experiments on this database to assess the correspondence between 16 mainstream AIGI quality models and human perception. We anticipate that this large-scale quality database will inspire robust quality indicators for AIGIs and propel the evolution of AIGC for vision. The database is released on https://www.modelscope.cn/datasets/lcysyzxdxc/AIGCQA-30K-Image.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Transactions on image processing, 27(1):206–219, 2017.
  2. Topiq: A top-down approach from semantics to distortions for image quality assessment, 2023.
  3. Pixart-α𝛼\alphaitalic_α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. 2310.00426, 2023.
  4. DeepFloyd. If-i-xl-v1.0. https://www.deepfloyd.ai, 2023.
  5. Cogview: Mastering text-to-image generation via transformers. Advances in Neural Information Processing Systems, 34:19822–19835, 2021.
  6. Light-vqa: A multi-dimensional quality assessment model for low-light video enhancement. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1088–1097, 2023.
  7. dreamlike art. dreamlike-photoreal-2.0. https://dreamlike.art, 2023.
  8. Vdpve: Vqa dataset for perceptual video enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1474–1483, 2023.
  9. Progressive knowledge distillation of stable diffusion xl using layer level loss. 2401.02677, 2024.
  10. David Holz. Midjourney. https://www.midjourney.com, 2023.
  11. The konstanz natural video database (konvid-1k). In 2017 Ninth international conference on quality of multimedia experience, pages 1–6. IEEE, 2017.
  12. Xgc-vqa: A unified video quality assessment model for user, professionally, and occupationally-generated content. In IEEE International Conference on Multimedia and Expo Workshops, 2023.
  13. Convolutional neural networks for no-reference image quality assessment. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1733–1740, 2014.
  14. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5148–5157, 2021.
  15. Adam: A method for stochastic optimization, 2014.
  16. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024.
  17. Stablevqa: A deep no-reference quality assessment model for video stability. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1066–1076, 2023.
  18. Subjective-aligned dataset and metric for text-to-video quality assessment, 2024.
  19. A pbch reception algorithm in 5g broadcasting. In IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, 2022.
  20. A real-time blind quality-of-experience assessment metric for http adaptive streaming. In IEEE International Conference on Multimedia and Expo, 2023.
  21. Misc: Ultra-low bitrate image semantic compression driven by large multimodal model, 2024.
  22. Q-refine: A perceptual quality refiner for ai-generated image, 2024.
  23. A full-reference quality assessment metric for cartoon images. In IEEE 24th International Workshop on Multimedia Signal Processing, 2022.
  24. Agiqa-3k: An open database for ai-generated image quality assessment. IEEE Transactions on Circuits and Systems for Video Technology, 2023.
  25. Lcm-lora: A universal stable-diffusion acceleration module, 2023.
  26. No-reference image quality assessment in the spatial domain. IEEE Transactions on image processing, 21(12):4695–4708, 2012.
  27. Making a “completely blind” image quality analyzer. IEEE Signal processing letters, 20(3):209–212, 2012.
  28. OpenAI. Gpt-4 technical report, 2023.
  29. PlaygroundAI. playground-v2-1024px-aesthetic. https://playground.com, 2023.
  30. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  31. Hierarchical text-conditional image generation with clip latents. 2204.06125, 2022.
  32. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
  33. Text-guided synthesis of artistic images with retrieval-augmented diffusion models. 2207.13038, 2022.
  34. Adversarial diffusion distillation, 2023.
  35. Blindly assess image quality in the wild guided by a self-adaptive hyper network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3667–3676, 2020.
  36. I. T. Union. Methodology for the subjective assessment of the quality of television pictures. ITU-R Recommendation BT. 500-11, 2002.
  37. Exploring clip for assessing the look and feel of images. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 2555–2563, 2023.
  38. Aigciqa2023: A large-scale image quality assessment database for ai generated images: from the perspectives of quality, authenticity and correspondence. In CAAI International Conference on Artificial Intelligence, pages 46–57. Springer, 2023.
  39. Q-bench: A benchmark for general-purpose foundation models on low-level vision, 2023.
  40. Q-instruct: Improving low-level visual abilities for multi-modality foundation models, 2023.
  41. Q-align: Teaching lmms for visual scoring via discrete text-defined levels, 2023.
  42. Towards open-ended visual quality comparison. arXiv preprint arXiv:2402.16641, 2024.
  43. Human preference score: Better aligning text-to-image models with human preference. pages 2096–2105, 2023.
  44. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024.
  45. Long-clip: Unlocking the long-text capability of clip, 2024.
  46. Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Transactions on Circuits and Systems for Video Technology, 30(1):36–47, 2018.
  47. Uncertainty-aware blind image quality assessment in the laboratory and wild. IEEE Transactions on Image Processing, 30:3474–3486, 2021.
  48. Blind image quality assessment via vision-language correspondence: A multitask learning perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14071–14081, 2023.
  49. A perceptual quality assessment exploration for aigc images. In IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pages 440–445, 2023.
  50. Gms-3dqa: Projection-based grid mini-patch sampling for 3d model quality assessment, 2023.
  51. Advancing zero-shot digital human quality assessment through text-prompted evaluation, 2023.
  52. Q-boost: On visual quality assessment ability of low-level multi-modality foundation models, 2023.
  53. A reduced-reference quality assessment metric for textured mesh digital humans. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024.
  54. Quality-of-experience evaluation for digital twins in 6g network environments. IEEE Transactions on Broadcasting, 2024.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (13)
  1. Chunyi Li (66 papers)
  2. Tengchuan Kou (8 papers)
  3. Yixuan Gao (17 papers)
  4. Yuqin Cao (16 papers)
  5. Wei Sun (373 papers)
  6. Zicheng Zhang (124 papers)
  7. Yingjie Zhou (43 papers)
  8. Zhichao Zhang (32 papers)
  9. Weixia Zhang (19 papers)
  10. Haoning Wu (68 papers)
  11. Xiaohong Liu (117 papers)
  12. Xiongkuo Min (138 papers)
  13. Guangtao Zhai (230 papers)
Citations (12)
View on Semantic Scholar