Can Knowledge Editing Really Correct Hallucinations? (2410.16251v2)

Abstract: LLMs suffer from hallucinations, referring to the non-factual information in generated content, despite their superior capacities across tasks. Meanwhile, knowledge editing has been developed as a new popular paradigm to correct the erroneous factual knowledge encoded in LLMs with the advantage of avoiding retraining from scratch. However, one common issue of existing evaluation datasets for knowledge editing is that they do not ensure LLMs actually generate hallucinated answers to the evaluation questions before editing. When LLMs are evaluated on such datasets after being edited by different techniques, it is hard to directly adopt the performance to assess the effectiveness of different knowledge editing methods in correcting hallucinations. Thus, the fundamental question remains insufficiently validated: Can knowledge editing really correct hallucinations in LLMs? We proposed HalluEditBench to holistically benchmark knowledge editing methods in correcting real-world hallucinations. First, we rigorously construct a massive hallucination dataset with 9 domains, 26 topics and more than 6,000 hallucinations. Then, we assess the performance of knowledge editing methods in a holistic way on five dimensions including Efficacy, Generalization, Portability, Locality, and Robustness. Through HalluEditBench, we have provided new insights into the potentials and limitations of different knowledge editing methods in correcting hallucinations, which could inspire future improvements and facilitate the progress in the field of knowledge editing.

Mitigating Hallucinations with Knowledge Editing: An Evaluation Framework

The paper "Mitigating Hallucinations with Knowledge Editing" addresses a critical issue in LLMs — hallucinations, defined as the generation of non-factual information. The authors focus on knowledge editing as a method to correct these hallucinations without retraining LLMs from scratch. They note a significant flaw in existing evaluation datasets which do not confirm whether LLMs produce hallucinated answers pre-edit, rendering the assessment of editing effectiveness questionable.

To tackle this, the authors present a comprehensive framework and benchmark to evaluate various knowledge editing methods on their ability to mitigate real-world hallucinations in LLMs. This paper systematically constructs a vast dataset with more than 6,000 confirmed hallucinations spanning 9 domains and 26 topics, providing a foundation for rigorous evaluation.

Key Contributions and Methodology

1. Dataset Construction: The authors meticulously curate a large-scale dataset from Wikipedia, ensuring that the knowledge triplets have a single truth, thus reliably establishing if a model provides hallucinated outputs.
2. Evaluation Dimensions:

The paper introduces a holistic assessment using five dimensions: - Efficacy: How well does the edited model correct hallucinations? - Generalization: Can the edited knowledge be applied to various related queries? - Portability: Does the edited knowledge transfer across logically connected facts? - Locality: Does the edit have minimal unintended effects on unrelated knowledge? - Robustness: Is the edited knowledge resistant to adversarial prompt alterations?

1. Knowledge Editing Techniques:

The paper assesses seven established methods: - Fine-tuning variations (FT-L, FT-M, LoRA) - Locate-then-edit methods (ROME, MEMIT) - In-context editing (ICE) - Memory-based adaptation (GRACE)

Findings

The paper reveals that performances reported on existing datasets may not be reliable indicators of a method's ability to correct hallucinations effectively. For instance, methods like FT-M and MEMIT, showing near-perfect performance on traditional datasets, underperformed on the proposed benchmark, indicating a disparity between theoretical efficacy and practical applicability.

• Efficacy: ICE and GRACE outperform others on Efficacy, though even they fall short outside controlled scenarios.
• Generalization and Portability: Most methods, except ICE marginally improve or worsen these scores, highlighting significant challenges.
• Locality and Robustness: FT-M and ICE excel in locality, yet robustness remains a challenge globally, with many models faltering under adversarial prompts.

Implications and Future Directions

This work provides crucial insights into the limitations and potential improvements needed in knowledge editing methods. It underscores the necessity for benchmarks that authentically simulate real-world errors to measure true effectiveness. The implications for AI development are substantial, as they guide researchers toward refining models that are both adaptive and reliable.

Future research could leverage these findings to enhance model architectures, refine editing algorithms, or potentially develop hybrid approaches combining strengths across methods. The robustness and locality of edits remain especially promising areas for exploration, aiming for model consistency and integrity without sacrificing responsiveness to corrections.

Overall, this paper contributes significantly to the discourse surrounding the mitigation of LLM hallucinations, pushing towards more dependable AI systems through strategic knowledge editing.