Investigating the Role of Prompting and External Tools in Hallucination Rates of Large Language Models

Abstract: LLMs are powerful computational models trained on extensive corpora of human-readable text, enabling them to perform general-purpose language understanding and generation. LLMs have garnered significant attention in both industry and academia due to their exceptional performance across various NLP tasks. Despite these successes, LLMs often produce inaccuracies, commonly referred to as hallucinations. Prompt engineering, the process of designing and formulating instructions for LLMs to perform specific tasks, has emerged as a key approach to mitigating hallucinations. This paper provides a comprehensive empirical evaluation of different prompting strategies and frameworks aimed at reducing hallucinations in LLMs. Various prompting techniques are applied to a broad set of benchmark datasets to assess the accuracy and hallucination rate of each method. Additionally, the paper investigates the influence of tool-calling agents (LLMs augmented with external tools to enhance their capabilities beyond language generation) on hallucination rates in the same benchmarks. The findings demonstrate that the optimal prompting technique depends on the type of problem, and that simpler techniques often outperform more complex methods in reducing hallucinations. Furthermore, it is shown that LLM agents can exhibit significantly higher hallucination rates due to the added complexity of external tool usage.

• The paper demonstrates that self-consistency (SC) outperforms chain-of-thought and other techniques in reducing hallucinations, notably on benchmarks like GSM8K.
• The paper shows that Chat Protect (CP) and real-time data integration via DDGA significantly lower factual hallucinations compared to knowledge graph methods.
• The paper highlights that incorporating external tools can increase complexity and hallucination rates, underscoring the need for careful strategy selection in LLM deployment.

Investigating Prompting and External Tools in LLM Hallucinations

The paper "Investigating the Role of Prompting and External Tools in Hallucination Rates of LLMs" explores methods to reduce inaccuracies, known as hallucinations, in LLMs. This research is pertinent due to the increasing deployment of LLMs across various applications, where hallucinations can lead to significant misinformation, particularly in sensitive domains like politics or medicine.

Hallucinations in LLMs

LLMs are known for their linguistic capabilities but suffer from hallucinations—outputs that are unfaithful to real-world facts or context. These hallucinations are categorized into factual and faithfulness hallucinations. Factual hallucinations include inconsistencies or unsupported fabrications, while faithfulness hallucinations involve deviations from prompt instructions or logic.

Prompt Engineering Techniques

The paper evaluates multiple prompting techniques designed to mitigate hallucinations:

1. Self-Consistency (SC): This technique employs majority voting across multiple samples to enhance reliability. It showed effectiveness particularly at a higher temperature (0.8) in the GSM8K benchmark, which involves mathematical reasoning.
2. Chain-of-Thought (CoT) and Tree-of-Thought (ToT): These approaches break down problem-solving into logical steps. The results indicate that while they improve reasoning, SC performed better due to its balance between creativity and accuracy.
3. Chat Protect (CP): By discarding answers where contradictions are detected among multiple samples, CP achieved the highest accuracy on the TriviaQA benchmark. Its performance improved at higher temperatures by reducing the number of hallucinations.
4. Knowledge Graph-based Retrofitting (KGR) and DuckDuckGo Augmentation (DDGA): The DDGA method, which adds real-time internet information to queries, improved the number of correct answers. However, incorporation of knowledge graphs like Wikidata in KGR didn't yield similar benefits due to implementation limitations.
5. Multiagent Debate (MAD): This interactive approach showed promising results, particularly on the MMLU benchmark with diverse subjects. The debate model allows for refining answers through agent interaction.
6. Reflection and Chain-of-Verification (CoVe): These methods involve critical feedback and verification questions to confirm the consistency of responses. CoVe resulted in high accuracy but at the expense of reduced question coverage.

Influence of External Tools

The study highlights that augmenting LLMs with external tools, such as through the ReAct framework, introduces complexity that can increase hallucination rates, particularly in less powerful models. The research underscores that simpler architectures often outperform more intricate setups due to reduced cognitive load on the model.

Implications and Future Directions

The findings of this study suggest that the effectiveness of mitigation strategies highly depends on the type of task. SC is particularly effective for reasoning tasks, while strategies like CP cater well to factual question settings. The introduction of external tools requires careful handling to avoid reducing performance quality.

Future research could explore combinations of strategies or dynamic adjustment techniques, such as adaptive temperature settings in SC, tailored to specific tasks. Moreover, the exploration of larger, more powerful models with external tools presents another avenue for research, potentially reducing the hallucination rates observed with smaller models.

This paper provides valuable insights into mitigating LLM hallucinations using prompting techniques and highlights the nuanced considerations when employing external tools in AI systems. The results significantly contribute to developing more reliable AI applications across a broad range of domains.