Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
120 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
51 tokens/sec
2000 character limit reached

An Audit on the Perspectives and Challenges of Hallucinations in NLP (2404.07461v2)

Published 11 Apr 2024 in cs.CL and cs.AI

Abstract: We audit how hallucination in LLMs is characterized in peer-reviewed literature, using a critical examination of 103 publications across NLP research. Through the examination of the literature, we identify a lack of agreement with the term `hallucination' in the field of NLP. Additionally, to compliment our audit, we conduct a survey with 171 practitioners from the field of NLP and AI to capture varying perspectives on hallucination. Our analysis calls for the necessity of explicit definitions and frameworks outlining hallucination within NLP, highlighting potential challenges, and our survey inputs provide a thematic understanding of the influence and ramifications of hallucination in society.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (132)
  1. Reducing named entity hallucination risk to ensure faithful summary generation. In Proceedings of the 16th International Natural Language Generation Conference, pages 437–442.
  2. Antonios Anastasopoulos and Graham Neubig. 2019. Pushing the limits of low-resource morphological inflection. arXiv preprint arXiv:1908.05838.
  3. Simon Baker and Takeo Kanade. 2000. Hallucinating faces. In Proceedings of 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG ’00), pages 83 – 88.
  4. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623.
  5. Melania Berbatova and Yoan Salambashev. 2023. Evaluating hallucinations in large language models for Bulgarian language. In Proceedings of the 8th Student Research Workshop associated with the International Conference Recent Advances in Natural Language Processing, pages 55–63, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.
  6. Qualitative HCI research: Going behind the scenes. Morgan & Claypool Publishers.
  7. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454–5476, Online. Association for Computational Linguistics.
  8. Nicholas Grant Boeving. 2020. Transpersonal psychology. In Encyclopedia of Psychology and Religion, pages 2392–2394. Springer.
  9. Adam Bouyamourn. 2023. Why LLMs hallucinate, and how to get (evidential) closure: Perceptual, intensional, and extensional learning for faithful natural language generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3181–3193, Singapore. Association for Computational Linguistics.
  10. Hallucinated but factual! inspecting the factuality of hallucinations in abstractive summarization. arXiv preprint arXiv:2109.09784.
  11. Exploring in-context learning for knowledge grounded dialog generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10071–10081.
  12. Improving faithfulness in abstractive summarization with contrast candidate generation and selection. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5935–5941, Online. Association for Computational Linguistics.
  13. Fidelity-enriched contrastive search: Reconciling the faithfulness-diversity trade-off in text generation. arXiv preprint arXiv:2310.14981.
  14. CaPE: Contrastive parameter ensembling for reducing hallucination in abstractive summarization. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10755–10773, Toronto, Canada. Association for Computational Linguistics.
  15. Holm: Hallucinating objects with language models for referring expression recognition in partially-observed scenes. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5440–5453.
  16. Robert Cooper and Michael Foster. 1971. Sociotechnical systems. American Psychologist, 26(5):467.
  17. Hallucinating law: Legal mistakes with large language models are pervasive.
  18. Plausible may not be faithful: Probing object hallucination in vision-language pre-training. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2136–2148, Dubrovnik, Croatia. Association for Computational Linguistics.
  19. Detecting and mitigating hallucinations in machine translation: Model internal workings alone do well, sentence similarity even better. arXiv preprint arXiv:2212.08597.
  20. HalOmi: A manually annotated benchmark for multilingual hallucination and omission detection in machine translation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 638–653, Singapore. Association for Computational Linguistics.
  21. Diving deep into modes of fact hallucinations in dialogue systems. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 684–699, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  22. Faithful to the document or to the world? mitigating hallucinations via entity-linked knowledge in abstractive summarization. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1067–1082, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  23. Faithdial: A faithful benchmark for information-seeking dialogue. Transactions of the Association for Computational Linguistics, 10:1473–1490.
  24. Scene graph as pivoting: Inference-time image-free unsupervised multimodal machine translation with visual scene hallucination. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5980–5994, Toronto, Canada. Association for Computational Linguistics.
  25. Towards opening the black box of neural machine translation: Source and target interpretations of the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8756–8769, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  26. Katja Filippova. 2020a. Controlled hallucinations: Learning to generate faithfully from noisy data. arXiv preprint arXiv:2010.05873.
  27. Katja Filippova. 2020b. Controlled hallucinations: Learning to generate faithfully from noisy data. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 864–870, Online. Association for Computational Linguistics.
  28. Creativity and psychopathology: are there similar mental processes involved in creativity and in psychosis-proneness? Frontiers in psychology, 5:117336.
  29. Lorenzo Jaime Flores and Arman Cohan. 2024. On the benefits of fine-grained loss truncation: A case study on factuality in summarization. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 138–150, St. Julian’s, Malta. Association for Computational Linguistics.
  30. Uncertainty aware review hallucination for science article classification. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5004–5009, Online. Association for Computational Linguistics.
  31. From melting pots to misrepresentations: Exploring harms in generative ai. arXiv preprint arXiv:2403.10776.
  32. Findings of the association for computational linguistics: Emnlp 2022. In Findings of the Association for Computational Linguistics: EMNLP 2022.
  33. Dealing with hallucination and omission in neural natural language generation: A use case on meteorology. In Proceedings of the 15th International Conference on Natural Language Generation, pages 121–130, Waterville, Maine, USA and virtual meeting. Association for Computational Linguistics.
  34. Dealing with hallucination and omission in neural natural language generation: A use case on meteorology. In Proceedings of the 15th International Conference on Natural Language Generation, pages 121–130.
  35. Optimal transport for unsupervised hallucination detection in neural machine translation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13766–13784, Toronto, Canada. Association for Computational Linguistics.
  36. Looking for a needle in a haystack: A comprehensive study of hallucinations in neural machine translation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1059–1075, Dubrovnik, Croatia. Association for Computational Linguistics.
  37. Sociodemographic bias in language models: A survey and forward path.
  38. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929–3938. PMLR.
  39. Face hallucination using bayesian global estimation and local basis selection. In 2010 IEEE International Workshop on Multimedia Signal Processing, pages 449–453.
  40. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232.
  41. Ann Irvine and Chris Callison-Burch. 2014. Hallucinating phrase translations for low resource mt. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 160–170.
  42. Tackling hallucinations in neural chart summarization. In Proceedings of the 16th International Natural Language Generation Conference, pages 414–423, Prague, Czechia. Association for Computational Linguistics.
  43. Tackling hallucinations in neural chart summarization. In Proceedings of the 16th International Natural Language Generation Conference, pages 414–423.
  44. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38.
  45. RHO: Reducing hallucination in open-domain dialogues with knowledge grounding. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4504–4522, Toronto, Canada. Association for Computational Linguistics.
  46. Towards mitigating llm hallucination via self reflection. In The 2023 Conference on Empirical Methods in Natural Language Processing.
  47. Towards mitigating LLM hallucination via self reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827–1843, Singapore. Association for Computational Linguistics.
  48. Embedding hallucination for few-shot language fine-tuning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5522–5530, Seattle, United States. Association for Computational Linguistics.
  49. Zhijing Jin and Rada Mihalcea. 2022. Natural language processing for policymaking. In Handbook of Computational Social Science for Policy, pages 141–162. Springer International Publishing Cham.
  50. Language models (mostly) know what they know.
  51. Andrej Karpathy. 2015. The unreasonable effectiveness of recurrent neural networks. https://karpathy.github.io/2015/05/21/rnn-effectiveness/.
  52. CRUSH4SQL: Collective retrieval using schema hallucination for Text2SQL. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14054–14066, Singapore. Association for Computational Linguistics.
  53. Contrastive error attribution for finetuned language models. arXiv preprint arXiv:2212.10722.
  54. When do pre-training biases propagate to downstream tasks? a case study in text summarization. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3206–3219.
  55. Mateusz Lango and Ondrej Dusek. 2023. Critic-driven decoding for mitigating hallucinations in data-to-text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2853–2862, Singapore. Association for Computational Linguistics.
  56. Scott H Lee. 2018. Natural language generation for electronic health records. NPJ digital medicine, 1(1):63.
  57. Lisa Legault. 2020. Encyclopedia of personality and individual differences. Encyclopedia of Personality and Individual Differences, pages 1–5.
  58. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.
  59. Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 292–305, Singapore. Association for Computational Linguistics.
  60. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334.
  61. Hui Liu and Xiaojun Wan. 2023. Models see hallucinations: Evaluating the factuality in video captioning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 11807–11823, Singapore. Association for Computational Linguistics.
  62. A token-level reference-free hallucination detection benchmark for free-form text generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6723–6737, Dublin, Ireland. Association for Computational Linguistics.
  63. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7052–7063, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  64. Zero-resource hallucination prevention for large language models.
  65. Fiona Macpherson and Dimitris Platchias. 2013. Hallucination: Philosophy and psychology. MIT Press.
  66. Eyes show the way: Modelling gaze behaviour for hallucination detection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11424–11438, Singapore. Association for Computational Linguistics.
  67. Open-world factually consistent question generation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 2390–2404, Toronto, Canada. Association for Computational Linguistics.
  68. Ai hallucinations: A misnomer worth clarifying. arXiv preprint arXiv:2401.06796.
  69. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017, Singapore. Association for Computational Linguistics.
  70. Andreas Marfurt and James Henderson. 2022. Unsupervised token-level hallucination detection from summary generation by-products. In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 248–261, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  71. Spontaneous and deliberate creative cognition during and after psilocybin exposure. Translational psychiatry, 11(1):209.
  72. How decoding strategies affect the verifiability of generated text. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 223–235, Online. Association for Computational Linguistics.
  73. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics.
  74. Chatgpt and bard exhibit spontaneous citation fabrication during psychiatry literature search. Psychiatry Research, 326:115334.
  75. Sources of hallucination by large language models on inference tasks. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2758–2774, Singapore. Association for Computational Linguistics.
  76. Reducing conversational agents’ overconfidence through linguistic calibration. Transactions of the Association for Computational Linguistics, 10:857–872.
  77. Beren Millidge. 2023. Llms confabulate not hallucinate.
  78. Domain robustness in neural machine translation. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 151–164, Virtual. Association for Machine Translation in the Americas.
  79. Entity-level factual consistency of abstractive text summarization. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2727–2733, Online. Association for Computational Linguistics.
  80. Pranav Narayanan Venkit. 2023. Towards a holistic approach: Understanding sociodemographic biases in nlp models using an interdisciplinary lens. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pages 1004–1005.
  81. Unmasking nationality bias: A study of human perception of nationalities in ai-generated articles. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pages 554–565.
  82. US News. 2023. Best national university rankings 2023.
  83. A simple recipe towards reducing hallucination in neural surface realisation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2673–2679, Florence, Italy. Association for Computational Linguistics.
  84. Med-HALT: Medical domain hallucination test for large language models. In Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), pages 314–334, Singapore. Association for Computational Linguistics.
  85. mmT5: Modular multilingual pre-training solves source language hallucinations. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1978–2008, Singapore. Association for Computational Linguistics.
  86. Mutual information alleviates hallucinations in abstractive summarization. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5956–5965, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  87. Improving graph-to-text generation using cycle training. In Proceedings of the 4th Conference on Language, Data and Knowledge, pages 256–261.
  88. Detecting and mitigating hallucinations in multilingual summarisation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8914–8932, Singapore. Association for Computational Linguistics.
  89. Invite: a testbed of automatically generated invalid questions to evaluate large language models for hallucinations. In The 2023 Conference on Empirical Methods in Natural Language Processing.
  90. The curious case of hallucinations in neural machine translation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1172–1183, Online. Association for Computational Linguistics.
  91. The troubling emergence of hallucination in large language models–an extensive definition, quantification, and prescriptive remediations. arXiv preprint arXiv:2310.04988.
  92. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922.
  93. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045, Brussels, Belgium. Association for Computational Linguistics.
  94. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 300–325, Online. Association for Computational Linguistics.
  95. DelucionQA: Detecting hallucinations in domain-specific question answering. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 822–835, Singapore. Association for Computational Linguistics.
  96. Farhan Samir and Miikka Silfverberg. 2022. One wug, two wug+s transformer inflection models hallucinate affixes. In Proceedings of the Fifth Workshop on the Use of Computational Methods in the Study of Endangered Languages, pages 31–40, Dublin, Ireland. Association for Computational Linguistics.
  97. Mitigating intrinsic named entity-related hallucinations of abstractive text summarization. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15807–15824, Singapore. Association for Computational Linguistics.
  98. Hallucination mitigation in natural language generation from large-scale open-domain knowledge graphs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12506–12521, Singapore. Association for Computational Linguistics.
  99. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784–3803, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  100. The curious case of hallucinatory (un) answerability: Finding truths in the hidden states of over-confident large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3607–3625.
  101. Human evaluation of conversations is an open problem: comparing the sensitivity of various methods for evaluating dialogue agents. In Proceedings of the 4th Workshop on NLP for Conversational AI, pages 77–97, Dublin, Ireland. Association for Computational Linguistics.
  102. HaRiM+{}^{+}start_FLOATSUPERSCRIPT + end_FLOATSUPERSCRIPT: Evaluating summary quality with hallucination risk. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 895–924, Online only. Association for Computational Linguistics.
  103. Brent J Steele. 2017. Hallucination and intervention. Global Discourse, 7(2-3):201–218.
  104. Towards fewer hallucinations in knowledge-grounded dialogue generation via augmentative and contrastive knowledge-dialogue. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1741–1750, Toronto, Canada. Association for Computational Linguistics.
  105. Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv preprint arXiv:2107.02137.
  106. Anirudh S. Sundar and Larry Heck. 2023. cTBLS: Augmenting large language models with conversational tables. In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 59–70, Toronto, Canada. Association for Computational Linguistics.
  107. Thematic analysis. The SAGE handbook of qualitative research in psychology, 2:17–37.
  108. Alberto Testoni and Raffaella Bernardi. 2021. “I’ve seen things you people wouldn’t believe”: Hallucinating entities in GuessWhat?! In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop, pages 101–111, Online. Association for Computational Linguistics.
  109. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5433–5442, Singapore. Association for Computational Linguistics.
  110. A comprehensive survey of hallucination mitigation techniques in large language models. arXiv preprint arXiv:2401.01313.
  111. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & health sciences, 15(3):398–405.
  112. The sentiment problem: A critical survey towards deconstructing sentiment analysis. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13743–13763.
  113. Overcoming catastrophic forgetting in zero-shot cross-lingual generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9279–9300, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  114. Contrastive decoding reduces hallucinations in large multilingual machine translation models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2526–2539, St. Julian’s, Malta. Association for Computational Linguistics.
  115. Chaojun Wang and Rico Sennrich. 2020. On exposure bias, hallucination and domain shift in neural machine translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3544–3552, Online. Association for Computational Linguistics.
  116. “according to . . . ”: Prompting language models improves quoting from pre-training data. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2288–2301, St. Julian’s, Malta. Association for Computational Linguistics.
  117. Stefan Werning. 2024. Generative ai and the technological imaginary of game design. In Creative Tools and the Softwarization of Cultural Production, pages 67–90. Springer.
  118. LaMini-LM: A diverse herd of distilled models from large-scale instructions. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 944–964, St. Julian’s, Malta. Association for Computational Linguistics.
  119. A brief overview of chatgpt: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5):1122–1136.
  120. Yijun Xiao and William Yang Wang. 2021a. On hallucination and predictive uncertainty in conditional language generation. arXiv preprint arXiv:2103.15025.
  121. Yijun Xiao and William Yang Wang. 2021b. On hallucination and predictive uncertainty in conditional language generation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2734–2744, Online. Association for Computational Linguistics.
  122. Fine-tuned LLMs know more, hallucinate less with few-shot sequence-to-sequence semantic parsing over Wikidata. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5778–5791, Singapore. Association for Computational Linguistics.
  123. A new benchmark and reverse validation method for passage-level hallucination detection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3898–3908, Singapore. Association for Computational Linguistics.
  124. Information-theoretic text hallucination reduction for video-grounded dialogue. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4182–4193, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  125. Automatic hallucination assessment for aligned large language models via transferable adversarial attacks.
  126. SAC33{}^{3}start_FLOATSUPERSCRIPT 3 end_FLOATSUPERSCRIPT: Reliable hallucination detection in black-box language models via semantic-aware cross-check consistency. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15445–15458, Singapore. Association for Computational Linguistics.
  127. Enhancing uncertainty-based hallucination detection with stronger focus. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 915–932, Singapore. Association for Computational Linguistics.
  128. Siren’s song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219.
  129. ERNIE: Enhanced language representation with informative entities. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1441–1451, Florence, Italy. Association for Computational Linguistics.
  130. Hallucination detection for grounded instruction generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4044–4053, Singapore. Association for Computational Linguistics.
  131. Why does chatgpt fall short in providing truthful answers?
  132. Detecting hallucinated content in conditional neural sequence generation. arXiv preprint arXiv:2011.02593.
Citations (2)
View on Semantic Scholar

Summary

  • The paper analyzes 103 peer-reviewed articles and surveys 171 practitioners to reveal inconsistencies in defining and measuring hallucinations in LLMs.
  • It highlights methodological challenges, noting the fragmented evaluation metrics and frameworks across various NLP applications.
  • The study recommends unifying terminology and standardizing documentation practices to address both technical and sociotechnical issues in LLM outputs.

Investigating the Phenomenon of Hallucination in LLMs

Introduction to Hallucination in NLP

LLMs have gained prominence in fields beyond traditional NLP applications, affecting sociotechnical systems and eliciting a mix of societal reactions. This paper scrutinizes the concept of `hallucination' within LLMs, analyzing 103 peer-reviewed articles and surveying 171 practitioners. Despite its growing relevance, the term suffers from vague interpretations, urging a need for explicit definitions and a unified framework. This analysis reveals the pervasive nature of hallucination across NLP tasks and highlights the discord in understanding and addressing it within the field.

The State of Hallucination Research

Hallucination in LLMs, initially linked to error generation like fabricating non-existent URLs, has evolved to denote inaccuracies in text outputs. Despite numerous attempts to mitigate such errors through technologies like self-checking models and Retrieval Augmented Generation Systems, the lack of consensus over its explicit definition remains a primary obstacle. This condition complicates efforts to measure, evaluate, and ultimately diminish hallucination effects, with methodologies ranging from human evaluation to statistical metrics, presenting a fragmented landscape with limited interoperability.

Societal Perceptions and Misconceptions

Exploring hallucination from a broader perspective uncovers a range of societal misconceptions. The comparison to human sensory experiences, for example, can mislead interpretations about the nature of errors made by LLMs, often correlating them with humanlike mental processes. This anthropomorphic view risks reinforcing stigmas around mental health and overlooks the purely data-driven origins of model inaccuracies. By reevaluating terminology and adopting terms that more accurately describe these phenomena, such as confabulation' orfabrication', the field might mitigate some of these societal misperceptions.

A Critical Look at Frameworks and Metrics in NLP Hallucination

The survey of literature and existing metrics underscores a critical gap in a standardized approach to defining and measuring hallucination within NLP. The disparity in frameworks across subfields, from conversational AI to machine translation, suggests a compartmentalized approach that hinders a cohesive understanding. Moreover, the current reliance on a broad suite of evaluation metrics, lacking in universal applicability and validation, further complicates the landscape.

Insights from the Practitioner Survey

The practitioner survey provides a grounded perspective on how hallucination is encountered and perceived in real-world settings. The majority of respondents acknowledge the phenomenon as a critical weakness of LLMs, with a significant portion encountering hallucination in their work. This contrast in the recognition of hallucination's impact, paired with diverse interpretations and preferred terminologies, echoes the need for a more unified and informed discourse within the community.

The Path Forward: Challenges and Recommendations

The paper concludes by delineating key challenges facing the field, including the absence of definitional clarity, measurement standardization, and the integration of sociotechnical frameworks. To address these, it proposes recommendations focused on adopting transparent documentation practices, standardizing terminology, and fostering community-centric discussions.

Closing Thoughts

The phenomenon of hallucination in LLMs represents a multifaceted challenge that extends beyond mere technical inaccuracies to entail broader sociotechnical implications. This paper's examination of the term across different research fronts and practitioner experiences offers a foundational step towards demystifying hallucination within the field of NLP. It underscores the importance of articulating a shared understanding that not only improves technological approaches but also aligns with ethical and societal considerations, ultimately enhancing the transparency, reliability, and acceptance of LLMs in various domains.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now