Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Strong hallucinations from negation and how to fix them (2402.10543v2)

Published 16 Feb 2024 in cs.CL and cs.AI

Abstract: Despite great performance on many tasks, LLMs (LMs) still struggle with reasoning, sometimes providing responses that cannot possibly be true because they stem from logical incoherence. We call such responses \textit{strong hallucinations} and prove that they follow from an LM's computation of its internal representations for logical operators and outputs from those representations. Focusing on negation, we provide a novel solution in which negation is treated not as another element of a latent representation, but as \textit{an operation over an LM's latent representations that constrains how they may evolve}. We show that our approach improves model performance in cloze prompting and natural language inference tasks with negation without requiring training on sparse negative data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Limits for learning with large language models. arXiv preprint arXiv:2306.12213.
  2. Nicholas Asher and Julie Hunter. 2022. When learning becomes impossible. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 107–116.
  3. Message exchange games in strategic conversations. Journal of Philosophical Logic, 46.4:355–404.
  4. Marco Baroni and Roberto Zamparelli. 2010. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of the 2010 conference on empirical methods in natural language processing, pages 1183–1193.
  5. Modal logic, Cambridge Tracts in Theoretical Computer Science No.53. Cambridge University Press.
  6. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326.
  7. Faithful to the original: Fact aware neural abstractive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
  8. Chen Chung Chang and H Jerome Keisler. 1973. Model theory. North Holland, Elsevier.
  9. Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answering. Computational Linguistics, pages 1–37.
  10. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pages 177–190. Springer.
  11. Bruno De Finetti. 1937. La prévision : ses lois logiques, ses sources subjectives. Annales de l’institut Henri Poincaré, 7:1–68.
  12. Semi-supervised multimodal representation learning through a global workspace. arXiv preprint arXiv:2306.15711.
  13. Bert: Pre-training of deep bidirectional transformers for language understanding.
  14. Handling divergent reference texts when evaluating table-to-text generation. arXiv preprint arXiv:1906.01081.
  15. Introduction to Montague semantics. Dordrecht. Synthese Library vol. 11.
  16. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378.
  17. Katja Filippova. 2020. Controlled hallucinations: Learning to generate faithfully from noisy data. arXiv preprint arXiv:2010.05873.
  18. Haim Gaifman. 1964. Concerning measures on boolean algebras. Pacific Journal of Mathematics, (14):61–73.
  19. Tanya Goyal and Greg Durrett. 2020. Evaluating factuality in generation with dependency-level entailment. arXiv preprint arXiv:2010.05478.
  20. Jeroen Groenendijk and Martin Stokhof. 1984. Studies on the Semantics of Questions and the Pragmatics of Answers. Ph.D. thesis, Centrale Interfaculteit, Amsterdam.
  21. Reto Gubelmann and Siegfried Handschuh. 2022. Context matters: A pragmatic study of plms’ negation understanding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4602–4621.
  22. Ned Hall. 1994. Correcting the guide to objective chance. Mind, 103(412):505–517.
  23. An analysis of natural language inference benchmarks through the lens of negation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9106–9118.
  24. Understanding by understanding not: Modeling negation in language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1301–1312.
  25. Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward. arXiv preprint arXiv:2005.01159.
  26. The factual inconsistency problem in abstractive text summarization: A survey. arXiv preprint arXiv:2104.14839.
  27. Beyond distributional hypothesis: Let language models learn meaning-text correspondence. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2030–2042.
  28. Becel: Benchmark for consistency evaluation of language models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3680–3696.
  29. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38.
  30. H. Kamp and U. Reyle. 1993. From Discourse to Logic: Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Kluwer Academic Publishers.
  31. Hans Kamp. 1981. A theory of truth and semantic representation. In J. Groenendijk, T. Janssen, and M. Stokhof, editors, Formal Methods in the Study of Language, pages 277–322. Mathematisch Centrum, Amsterdam.
  32. Lauri Karttunen and Stanley Peters. 1979. Conventional lmplicature. In Presupposition, pages 1–56. Brill.
  33. Nora Kassner and Hinrich Schütze. 2020. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7811–7818.
  34. Probing structural constraints of negation in pretrained language models. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 541–554.
  35. Hugues Leblanc. 1979. Probabilistic semantics for first-order logic. Zeitschrift für mathematische Logik und Grundlagen der Mathematik, 25(32):497–509. Wiley Online Library.
  36. David Lewis. 1981. A subjectivist’s guide to objective chance. In IFS: Conditionals, Belief, Decision, Chance and Time, pages 267–297. Springer.
  37. Implicit representations of meaning in neural language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1813–1827.
  38. HaluEval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449–6464, Singapore. Association for Computational Linguistics.
  39. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557.
  40. Roberta: A robustly optimized bert pretraining approach.
  41. Entity-based knowledge conflicts in question answering. arXiv preprint arXiv:2109.05052.
  42. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32.
  43. Entailment semantics can be extracted from an ideal language model. arXiv preprint arXiv:2209.12407.
  44. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332.
  45. Totto: A controlled table-to-text generation dataset. arXiv preprint arXiv:2004.14373.
  46. Language models as knowledge bases? In In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019.
  47. Frank Plumpton Ramsey. 1931. The foundations of mathematics and other logical essays. K. Paul, Trench, Trubner & Company, Limited.
  48. CoQA: A Conversational Question Answering Challenge. Transactions of the Association for Computational Linguistics, 7:249–266.
  49. M. Rooth. 1992. A theory of focus interpretation. Natural Language Semantics, 1(1):75–116.
  50. Bleurt: Learning robust metrics for text generation. arXiv preprint arXiv:2004.04696.
  51. olmpics-on what language model pre-training captures. Transactions of the Association for Computational Linguistics, 8:743–758.
  52. Alfred Tarski. 1944. The semantic conception of truth: and the foundations of semantics. Philosophy and phenomenological research, 4(3):341–376.
  53. What do you learn from context? probing for sentence structure in contextualized word representations. In International Conference on Learning Representations.
  54. Language models are not naysayers: An analysis of language models on negation benchmarks. arXiv preprint arXiv:2306.08189.
  55. Representing discourse in context. In Johan van Benthem and Alice ter Meulen, editors, Handbook of Logic and Linguistics, pages 179–237. Elsevier.
  56. Rufin VanRullen and Ryota Kanai. 2021. Deep learning and the global workspace theory. Trends in Neurosciences, 44(9):692–704.
  57. Attention is all you need. Advances in neural information processing systems, 30.
  58. Chain-of-thought prompting elicits reasoning in large language models.
  59. Zining Zhu and Frank Rudzicz. 2020. An information theoretic view on selecting linguistic probes. arXiv preprint arXiv:2009.07364.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets