Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
72 tokens/sec
GPT-4o
61 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Assessment of Multimodal Large Language Models in Alignment with Human Values (2403.17830v1)

Published 26 Mar 2024 in cs.CV

Abstract: LLMs aim to serve as versatile assistants aligned with human values, as defined by the principles of being helpful, honest, and harmless (hhh). However, in terms of Multimodal LLMs (MLLMs), despite their commendable performance in perception and reasoning tasks, their alignment with human values remains largely unexplored, given the complexity of defining hhh dimensions in the visual world and the difficulty in collecting relevant data that accurately mirrors real-world situations. To address this gap, we introduce Ch3Ef, a Compreh3ensive Evaluation dataset and strategy for assessing alignment with human expectations. Ch3Ef dataset contains 1002 human-annotated data samples, covering 12 domains and 46 tasks based on the hhh principle. We also present a unified evaluation strategy supporting assessment across various scenarios and different perspectives. Based on the evaluation results, we summarize over 10 key findings that deepen the understanding of MLLM capabilities, limitations, and the dynamic relationships between evaluation levels, guiding future advancements in the field.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (105)
  1. Aligning large language models with human: A survey. arXiv preprint arXiv:2307.12966, 2023a.
  2. Iason Gabriel. Artificial intelligence, values, and alignment. Minds and Machines, 30(3):411–437, 2020.
  3. In conversation with artificial intelligence: Aligning language models with human values. Philosophy & Technology, 36(2):27, 2023.
  4. Training language models to follow instructions with human feedback. NeurIPS, 35:27730–27744, 2022.
  5. Preference ranking optimization for human alignment. arXiv preprint arXiv:2306.17492, 2023.
  6. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021.
  7. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
  8. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
  9. Visual instruction tuning. In NeurIPS, 2023a.
  10. Instructblip: Towards general-purpose vision-language models with instruction tuning. In NeurIPS, 2023a.
  11. LAMM: language-assisted multi-modal instruction-tuning dataset, framework, and benchmark. In NeurIPS, 2023.
  12. From gpt-4 to gemini and beyond: Assessing the landscape of mllms on generalizability, trustworthiness and causality through four modalities. arXiv preprint arXiv:2401.15071, 2024.
  13. Safety of multimodal large language models on images and text. arXiv preprint arXiv:2402.00357, 2024.
  14. Query-relevant images jailbreak large multi-modal models. arXiv preprint arXiv:2311.17600, 2023b.
  15. Beyond task performance: Evaluating and reducing the flaws of large multimodal models with in-context learning. arXiv preprint arXiv:2310.00647, 2023.
  16. Evaluating object hallucination in large vision-language models. In EMNLP 2023, pages 292–305, 2023a.
  17. Seed-bench-2: Benchmarking multimodal large language models. arXiv preprint arXiv:2311.17092, 2023b.
  18. Vila: On pre-training for visual language models. arXiv preprint arXiv:2312.07533, 2023.
  19. Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving. arXiv preprint arXiv:2312.09245, 2023b.
  20. Unified hallucination detection for multimodal large language models. arXiv preprint arXiv:2402.03190, 2024.
  21. Visually dehallucinative instruction generation: Know what you don’t know. arXiv preprint arXiv:2402.09717, 2024.
  22. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022.
  23. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems, 36, 2024a.
  24. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023.
  25. Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281, 2023c.
  26. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394, 2023.
  27. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. arXiv preprint arXiv:2311.16502, 2023.
  28. Lvlm-ehub: A comprehensive evaluation benchmark for large vision-language models. arXiv preprint arXiv:2306.09265, 2023.
  29. Learn to explain: Multimodal reasoning via thought chains for science question answering. In NeurIPS, 2022.
  30. Obtaining well calibrated probabilities using bayesian binning. In AAAI, pages 2901–2907, 2015.
  31. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  32. Language models are few-shot learners. In NeurIPS, pages 1877–1901, 2020.
  33. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
  34. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023), 2023.
  35. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, volume 202, pages 19730–19742, 2023c.
  36. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023.
  37. InternLM Team. Internlm: A multilingual language model with progressively enhanced capabilities. https://github.com/InternLM/InternLM, 2023. Accessed: 2023-12-26.
  38. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. arXiv preprint arXiv:2312.00849, 2023.
  39. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525, 2023.
  40. Learning to summarize with human feedback. NeurIPS, 33:3008–3021, 2020.
  41. Hod: A benchmark dataset for harmful object detection. arXiv preprint arXiv:2310.05192, 2023.
  42. Goat-bench: Safety insights to large multimodal models through meme-based social abuse. arXiv preprint arXiv:2401.01523, 2024.
  43. Safety fine-tuning at (almost) no cost: A baseline for vision large language models. arXiv preprint arXiv:2402.02207, 2024.
  44. Direct preference optimization: Your language model is secretly a reward model. NeurIPS, 36, 2024.
  45. Safe rlhf: Safe reinforcement learning from human feedback. arXiv preprint arXiv:2310.12773, 2023b.
  46. A minimaximalist approach to reinforcement learning from human feedback. arXiv preprint arXiv:2401.04056, 2024.
  47. Aligner: Achieving efficient alignment through weak-to-strong correction. arXiv preprint arXiv:2402.02416, 2024b.
  48. Red-teaming large language models using chain of utterances for safety-alignment. arXiv preprint arXiv:2308.09662, 2023.
  49. Halle-switch: Controlling object hallucination in large vision language models. arXiv e-prints, pages arXiv–2310, 2023.
  50. Gaia: a benchmark for general ai assistants. arXiv preprint arXiv:2311.12983, 2023.
  51. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359, 2021.
  52. Sociotechnical safety evaluation of generative ai systems. arxiv, 2023.
  53. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:2401.05561, 2024.
  54. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models. arXiv preprint arXiv:2402.05044, 2024.
  55. OpenAI. Usage policies. Website, 2024. https://openai.com/policies/usage-policies.
  56. Meta. Use policies. Website, 2024. https://ai.meta.com/llama/use-policy/.
  57. Google. Use policies. Website, 2024. https://policies.google.com/terms/generative-ai/use-policy.
  58. NIST AI. Artificial intelligence risk management framework (ai rmf 1.0). 2023.
  59. Jessica Newman. A taxonomy of trustworthiness for artificial intelligence. CLTC: North Charleston, SC, USA, 1, 2023.
  60. Decodingtrust: A comprehensive assessment of trustworthiness in GPT models. In NeurIPS, 2023c.
  61. Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. arXiv preprint arXiv:2308.05374, 2023d.
  62. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110, 2022.
  63. Michel Cannarsa. Ethics guidelines for trustworthy ai. The Cambridge handbook of lawyering in the digital age, pages 283–297, 2021.
  64. On fairness and calibration. NIPS, 30, 2017.
  65. A survey of safety and trustworthiness of large language models through the lens of verification and validation. arXiv preprint arXiv:2305.11391, 2023.
  66. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot. In Towards Generalist Robots: Learning Paradigms for Scalable Skill Acquisition@ CoRL2023, 2023.
  67. OpenICL: An open-source framework for in-context learning. In ACL, 2023.
  68. OpenNMT: Open-source toolkit for neural machine translation. In ACL, 2017.
  69. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923, 2023.
  70. Can large language models be an alternative to human evaluations? In ACL, pages 15607–15631, 2023.
  71. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744, 2023e.
  72. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023.
  73. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. arXiv preprint arXiv:2311.04257, 2023.
  74. Llama-adapter v2: Parameter-efficient visual instruction model. arXiv preprint arXiv:2304.15010, 2023.
  75. Otter: A multi-modal model with in-context instruction tuning. arXiv preprint arXiv:2305.03726, 2023d.
  76. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824, 2023.
  77. Shikra: Unleashing multimodal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195, 2023.
  78. A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4), 2009.
  79. Benchmarking omni-vision representation through the lens of visual realms. In ECCV, pages 594–611, 2022a.
  80. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html, 2012.
  81. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, pages 67–78, 2014.
  82. Learning to count everything. In CVPR, pages 3394–3403, 2021.
  83. A dataset of clinically generated visual questions and answers about radiology images. Scientific Data, 5(1):180251, Nov 2018. ISSN 2052-4463. doi: 10.1038/sdata.2018.251. URL https://doi.org/10.1038/sdata.2018.251.
  84. Radiopaedia. Radiopaedia. Website, 2024. https://radiopaedia.org/.
  85. Rsvqa: Visual question answering for remote sensing data. IEEE Transactions on Geoscience and Remote Sensing, 58(12):8555–8566, December 2020. ISSN 1558-0644. doi: 10.1109/tgrs.2020.2988782. URL http://dx.doi.org/10.1109/TGRS.2020.2988782.
  86. Cartoon face recognition: A benchmark dataset. In Proceedings of the 28th ACM International Conference on Multimedia, pages 2264–2272, 2020.
  87. Towards vqa models that can read. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8317–8326, 2019.
  88. Slidevqa: A dataset for document visual question answering on multiple images. In AAAI, 2023.
  89. End-to-end scene text recognition. In 2011 International conference on computer vision, pages 1457–1464. IEEE, 2011.
  90. Uncertainty-based traffic accident anticipation with spatio-temporal relational learning. In ACM Multimedia Conference, May 2020.
  91. Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation. arXiv preprint, 2023.
  92. What one intelligence test measures: A theoretical account of the processing in the raven progressive matrices test. Psychological review, 97:404–31, 07 1990. doi: 10.1037/0033-295X.97.3.404.
  93. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, 15(2018):11, 2018.
  94. Microsoft coco: Common objects in context, Jan 2014. URL http://dx.doi.org/10.1007/978-3-319-10602-1_48.
  95. Privacyalert: A dataset for image privacy prediction. Proceedings of the International AAAI Conference on Web and Social Media, 16(1):1352–1361, May 2022. doi: 10.1609/icwsm.v16i1.19387. URL https://ojs.aaai.org/index.php/ICWSM/article/view/19387.
  96. Laion-5b: An open large-scale dataset for training next generation image-text models, 2022.
  97. The hateful memes challenge: Detecting hate speech in multimodal memes, 2021.
  98. Tovilag: Your visual-language generative model is also an evildoer, 2023d.
  99. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, pages 3354–3361, 2012.
  100. nuscenes: A multimodal dataset for autonomous driving. arXiv preprint arXiv:1903.11027, 2019.
  101. Psysafe: A comprehensive framework for psychological-based attack, defense, and evaluation of multi-agent system safety, 2024.
  102. What makes good in-context examples for gpt-3? In DeeLIO 2022, 2022. doi: 10.18653/v1/2022.deelio-1.10.
  103. Selective annotation makes language models better few-shot learners. In ICLR, 2023.
  104. Seed-bench: Benchmarking multimodal llms with generative comprehension. arXiv preprint arXiv:2307.16125, 2023e.
  105. Bamboo: Building mega-scale vision dataset continually with human-machine synergy. arXiv preprint arXiv:2203.07845, 2022b.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (10)
  1. Zhelun Shi (9 papers)
  2. Zhipin Wang (5 papers)
  3. Hongxing Fan (6 papers)
  4. Zaibin Zhang (6 papers)
  5. Lijun Li (30 papers)
  6. Yongting Zhang (4 papers)
  7. Zhenfei Yin (41 papers)
  8. Lu Sheng (63 papers)
  9. Yu Qiao (563 papers)
  10. Jing Shao (109 papers)
Citations (9)
View on Semantic Scholar