Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectification (2311.09114v2)

Published 15 Nov 2023 in cs.CL, cs.AI, and cs.LG

Abstract: LLMs have demonstrated remarkable proficiency in generating fluent text. However, they often encounter the challenge of generating inaccurate or hallucinated content. This issue is common in both non-retrieval-based generation and retrieval-augmented generation approaches, and existing post-hoc rectification methods may not address the accumulated hallucination errors that may be caused by the "snowballing" issue, especially in reasoning tasks. To tackle these challenges, we introduce a novel approach called Real-time Verification and Rectification (Ever). Instead of waiting until the end of the generation process to rectify hallucinations, Ever employs a real-time, step-wise generation and hallucination rectification strategy. The primary objective is to detect and rectify hallucinations as they occur during the text generation process. When compared to both retrieval-based and non-retrieval-based baselines, Ever demonstrates a significant improvement in generating trustworthy and factually accurate text across a diverse range of tasks, including short-form QA, biography generation, and multi-hop reasoning.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Do language models know when they’re hallucinating references? arXiv preprint arXiv:2305.18248.
  2. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511.
  3. Ralph Allan Bradley and Milton E Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345.
  4. Factool: Factuality detection in generative ai–a tool augmented framework for multi-task and multi-domain scenarios. arXiv preprint arXiv:2307.13528.
  5. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883.
  6. Lm vs lm: Detecting factual errors via cross examination. arXiv preprint arXiv:2305.13281.
  7. Chain-of-verification reduces hallucination in large language models. arXiv preprint arXiv:2309.11495.
  8. Halo: Estimation and reduction of hallucinations in open-source weak large language models. arXiv preprint arXiv:2308.11764.
  9. Attributed text generation via post-hoc research and revision. arXiv preprint arXiv:2210.08726.
  10. Pal: Program-aided language models. In International Conference on Machine Learning, pages 10764–10799. PMLR.
  11. Enabling large language models to generate text with citations. arXiv preprint arXiv:2305.14627.
  12. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738.
  13. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38.
  14. Active retrieval augmented generation. arXiv preprint arXiv:2305.06983.
  15. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551.
  16. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221.
  17. Large language models struggle to learn long-tail knowledge. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 15696–15707. PMLR.
  18. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213.
  19. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.
  20. Inference-time intervention: Eliciting truthful answers from a language model. arXiv preprint arXiv:2306.03341.
  21. Contrastive decoding: Open-ended text generation as optimization. arXiv preprint arXiv:2210.15097.
  22. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896.
  23. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147.
  24. Reducing conversational agents’ overconfidence through linguistic calibration. Transactions of the Association for Computational Linguistics, 10:857–872.
  25. A discrete hard EM approach for weakly supervised question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2851–2864, Hong Kong, China. Association for Computational Linguistics.
  26. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. arXiv preprint arXiv:2305.14251.
  27. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
  28. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813.
  29. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290.
  30. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761.
  31. Trusting your evidence: Hallucinate less with context-aware decoding. arXiv preprint arXiv:2305.14739.
  32. Head-to-tail: How knowledgeable are large language models (llm)? aka will llms replace knowledge graphs? arXiv preprint arXiv:2308.10168.
  33. Fine-tuning language models for factuality. arXiv preprint arXiv:2311.08401.
  34. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  35. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509.
  36. A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation. arXiv preprint arXiv:2307.03987.
  37. Freshllms: Refreshing large language models with search engine augmentation. arXiv preprint arXiv:2310.03214.
  38. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  39. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  40. " according to…" prompting language models improves quoting from pre-training data. arXiv preprint arXiv:2305.13252.
  41. Generating natural language proofs with verifier-guided search. arXiv preprint arXiv:2205.12443.
  42. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600.
  43. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.
  44. Improving language models via plug-and-play retrieval feedback. arXiv preprint arXiv:2305.14002.
  45. How language model hallucinations can snowball. arXiv preprint arXiv:2305.13534.
  46. Siren’s song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219.
  47. Verify-and-edit: A knowledge-enhanced chain-of-thought framework. arXiv preprint arXiv:2305.03268.
  48. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625.
  49. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Haoqiang Kang (7 papers)
  2. Juntong Ni (10 papers)
  3. Huaxiu Yao (103 papers)
Citations (27)
View on Semantic Scholar