2000 character limit reached
Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators (2310.07289v1)
Published 11 Oct 2023 in cs.CL
Abstract: LLMs outperform information retrieval techniques for downstream knowledge-intensive tasks when being prompted to generate world knowledge. However, community concerns abound regarding the factuality and potential implications of using this uncensored knowledge. In light of this, we introduce CONNER, a COmpreheNsive kNowledge Evaluation fRamework, designed to systematically and automatically evaluate generated knowledge from six important perspectives -- Factuality, Relevance, Coherence, Informativeness, Helpfulness and Validity. We conduct an extensive empirical analysis of the generated knowledge from three different types of LLMs on two widely studied knowledge-intensive tasks, i.e., open-domain question answering and knowledge-grounded dialogue. Surprisingly, our study reveals that the factuality of generated knowledge, even if lower, does not significantly hinder downstream tasks. Instead, the relevance and coherence of the outputs are more important than small factual mistakes. Further, we show how to use CONNER to improve knowledge-intensive tasks by designing two strategies: Prompt Engineering and Knowledge Selection. Our evaluation code and LLM-generated knowledge with human annotations will be released to facilitate future research.
- GPT-Neo: Large scale autoregressive language modeling with Mesh-Tensorflow.
- Virginia Braun and Victoria Clarke. 2012. Thematic analysis., pages 57–71.
- Towards robust personalized dialogue generation via order-insensitive representation regularization. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7337–7345, Toronto, Canada. Association for Computational Linguistics.
- Goal awareness for conversational AI: proactivity, non-collaborativity, and beyond. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, ACL 2023, Toronto, Canada, July 9-14, 2023, pages 1–10. Association for Computational Linguistics.
- Knowledge-enhanced mixed-initiative dialogue system for emotional support conversations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, pages 4079–4095. Association for Computational Linguistics.
- Wizard of wikipedia: Knowledge-powered conversational agents. arXiv preprint arXiv:1811.01241.
- Faithdial: A faithful benchmark for information-seeking dialogue.
- Revisiting text decomposition methods for nli-based factuality scoring of summaries. CoRR, abs/2211.16853.
- Revisiting text decomposition methods for nli-based factuality scoring of summaries.
- q2superscript𝑞2q^{2}italic_q start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT: Evaluating factual consistency in knowledge-grounded dialogues via question generation and question answering.
- OCNLI: Original Chinese Natural Language Inference. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3512–3526, Online. Association for Computational Linguistics.
- A discourse coherence model for analyzing chinese students’ essay. In 2017 International Conference on Progress in Informatics and Computing (PIC), pages 430–434.
- Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open domain question answering. In EACL 2021, pages 874–880.
- Query expansion by prompting large language models. CoRR, abs/2305.03653.
- Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12).
- Rethinking self-supervision objectives for generalizable coherence modeling. CoRR, abs/2110.07198.
- Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221.
- Language models (mostly) know what they know. CoRR, abs/2207.05221.
- Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781.
- Internet-augmented dialogue generation.
- Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9332–9346, Online. Association for Computational Linguistics.
- Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, pages 9332–9346.
- Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452–466.
- Factuality enhanced language models for open-ended text generation.
- Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.
- Halueval: A large-scale hallucination evaluation benchmark for large language models. CoRR, abs/2305.11747.
- Eliciting knowledge from large pre-trained models for unsupervised knowledge-grounded conversation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, pages 10551–10564.
- Evaluating verifiability in generative search engines.
- On learning to summarize with large language models as references. CoRR, abs/2305.14239.
- Multi-stage prompting for knowledgeable dialogue generation.
- Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. CoRR, abs/2303.08896.
- On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pages 1906–1919.
- Factscore: Fine-grained atomic evaluation of factual precision in long form text generation.
- Multi-stage document ranking with BERT. CoRR, abs/1910.14424.
- OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
- Training language models to follow instructions with human feedback.
- Understanding factuality in abstractive summarization with FRANK: A benchmark for factuality metrics. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4812–4829, Online. Association for Computational Linguistics.
- Fact-checking complex claims with program-guided reasoning.
- Check your facts and try again: Improving large language models with external knowledge and automated feedback.
- Kilt: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2523–2544.
- Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Exploring the limits of transfer learning with a unified text-to-text transformer.
- Squad: 100,000+ questions for machine comprehension of text.
- Colbertv2: Effective and efficient retrieval via lightweight late interaction. CoRR, abs/2112.01488.
- Get your vitamin C! robust fact verification with contrastive evidence. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 624–643, Online. Association for Computational Linguistics.
- Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3784–3803, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Robert H Somers. 1962. A new asymmetric measure of association for ordinal variables. American sociological review, pages 799–811.
- FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809–819, New Orleans, Louisiana. Association for Computational Linguistics.
- Llama: Open and efficient foundation language models.
- Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550, Online. Association for Computational Linguistics.
- Asking and answering questions to evaluate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5008–5020, Online. Association for Computational Linguistics.
- GPT-NER: named entity recognition via large language models. CoRR, abs/2304.10428.
- Finetuned language models are zero-shot learners.
- T2ranking: A large-scale chinese benchmark for passage ranking.
- Generate rather than retrieve: Large language models are strong context generators.
- Siren’s song in the ai ocean: A survey on hallucination in large language models.
Collections
Sign up for free to add this paper to one or more collections.
