Overconfidence is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Models (2405.02917v1)
Abstract: Language and Vision-LLMs (LLMs/VLMs) have revolutionized the field of AI by their ability to generate human-like text and understand images, but ensuring their reliability is crucial. This paper aims to evaluate the ability of LLMs (GPT4, GPT-3.5, LLaMA2, and PaLM 2) and VLMs (GPT4V and Gemini Pro Vision) to estimate their verbalized uncertainty via prompting. We propose the new Japanese Uncertain Scenes (JUS) dataset, aimed at testing VLM capabilities via difficult queries and object counting, and the Net Calibration Error (NCE) to measure direction of miscalibration. Results show that both LLMs and VLMs have a high calibration error and are overconfident most of the time, indicating a poor capability for uncertainty estimation. Additionally we develop prompts for regression tasks, and we show that VLMs have poor calibration when producing mean/standard deviation and 95% confidence intervals.
- Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
- Palm 2 technical report. arXiv preprint arXiv:2305.10403.
- A close look into the calibration of pre-trained language models. arXiv preprint arXiv:2211.00151.
- Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
- On calibration of modern neural networks. In International conference on machine learning, pages 1321–1330. PMLR.
- A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232.
- Look before you leap: An exploratory study of uncertainty measurement for large language models. arXiv preprint arXiv:2307.10236.
- How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962–977.
- Uncertainty-aware evaluation for vision-language models. arXiv preprint arXiv:2402.14418.
- OpenAI. 2023. Gpt-4v(ision) system card.
- Martino Pelucchi. 2023. Exploring chatgpt’s accuracy and confidence in high-resource languages.
- Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–1642, Seattle, Washington, USA. Association for Computational Linguistics.
- Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
- Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142–147.
- Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
- Matias Valdenegro-Toro. 2021. I find your lack of uncertainty in computer vision disturbing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 1263–1272.
- Sean Wallis. 2013. Binomial confidence intervals and contingency tests: mathematical fundamentals and the evaluation of alternative methods. Journal of Quantitative Linguistics, 20(3):178–208.
- Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
- Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. arXiv preprint arXiv:2306.13063.
- Tobias Groot (1 paper)
- Matias Valdenegro-Toro (62 papers)