Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks

Abstract: Current approaches of knowledge editing struggle to effectively propagate updates to interconnected facts. In this work, we delve into the barriers that hinder the appropriate propagation of updated knowledge within these models for accurate reasoning. To support our analysis, we introduce a novel reasoning-based benchmark -- ReCoE (Reasoning-based Counterfactual Editing dataset) -- which covers six common reasoning schemes in real world. We conduct a thorough analysis of existing knowledge editing techniques, including input augmentation, finetuning, and locate-and-edit. We found that all model editing methods show notably low performance on this dataset, especially in certain reasoning schemes. Our analysis over the chain-of-thought generation of edited models further uncover key reasons behind the inadequacy of existing knowledge editing methods from a reasoning standpoint, involving aspects on fact-wise editing, fact recall ability, and coherence in generation. We will make our benchmark publicly available.

• The paper presents ReCoE, a benchmark that evaluates how effectively edited facts propagate through reasoning in LLMs.
• It compares methods like input-augmentation, QLoRA, and MEMIT, showing that current approaches largely fail in coherent multi-step reasoning.
• Findings emphasize the need for improved generalization and logical coherence in knowledge editing to support dynamic factual updates.

Reasoning-Based Assessment of Knowledge Editing: Insights from ReCoE

Introduction

The paper "Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks" (2401.17585) presents a comprehensive evaluation of knowledge editing methods in LLMs, focusing on their ability to propagate updates to interconnected facts and support coherent reasoning. The authors introduce ReCoE, a novel benchmark designed to assess counterfactual knowledge editing across six reasoning schemes, and systematically analyze the limitations of current editing approaches, including input-augmentation, finetuning (QLoRA), and locate-and-edit (MEMIT).

Motivation and Problem Statement

While LLMs encode vast factual knowledge, their ability to update and propagate new information remains limited, especially when reasoning over interconnected facts. Existing editing methods often succeed at direct fact recall but fail to support multi-step reasoning with edited knowledge, as illustrated in reasoning-based assessments.

Figure 1: Reasoning-based assessment reveals that existing methods answer edited facts but fail at reasoning with them.

The paper identifies three critical competencies for effective knowledge propagation post-editing: (1) fact-wise editing effectiveness, (2) fact recall accuracy, and (3) logical coherence in generation. These dimensions are essential for robust knowledge editing in real-world applications.

ReCoE Benchmark: Construction and Characteristics

ReCoE is constructed using a hybrid-synthetic approach, combining existing QA datasets and LLM-assisted data synthesis. It covers six reasoning schemes: superlative, comparative, sorting, counting, aggregation, and subtraction. Each datapoint includes a question, answer, supporting facts, counterfactual answer, and counterfactual facts, enabling rigorous evaluation of knowledge propagation.

Figure 2: Step-by-step demonstration of ReCoE dataset construction, including data sourcing, generation, and counterfactual creation.

Unlike prior benchmarks that rely on synthetic or triplet-based fact representations, ReCoE employs OpenIE-style facts, introducing greater complexity and ambiguity, which better reflects real-world scenarios.

Figure 3: Comparison of fact representations in MQuAKE (triplet-based) and ReCoE (OpenIE-style), highlighting increased complexity in ReCoE.

Experimental Setup

The authors evaluate three representative knowledge editing methods on the Tülu series (Llama-based instruction-tuned models):

• Input-augmentation: Appends counterfactual facts to the prompt at inference time (upper bound).
• Finetuning (QLoRA): Parameter-efficient finetuning on new facts.
• Locate-and-edit (MEMIT): Directly edits feedforward modules in transformer layers to insert new facts.

Performance is measured using the correct_flip metric (percentage of predictions that transition from the original to the counterfactual answer) and is further analyzed via chain-of-thought (CoT) prompting.

Results and Analysis

Knowledge Probing and Editing Performance

Both model scaling and CoT prompting improve baseline QA accuracy. However, after editing, input-augmentation remains the most effective, while QLoRA and MEMIT exhibit substantial deficits in propagating knowledge for reasoning tasks.

• Input-augmentation achieves reasonable performance, but struggles with aggregation and subtraction (<50% accuracy).
• QLoRA shows moderate improvement with CoT and scaling, but overall performance is significantly lower than input-augmentation.
• MEMIT consistently underperforms, with near-zero accuracy in several reasoning schemes and severe degradation in generation coherence.

Fact-wise Editing Effectiveness

Fact-wise editing is assessed via perplexity over factual and counterfactual sentences. QLoRA demonstrates effective editing (lower PPL for counterfactuals post-edit), while MEMIT increases overall perplexity, indicating ineffective edits.

Figure 4: Fact-wise perplexity comparison before and after editing with QLoRA and MEMIT (7B). QLoRA achieves effective edits; MEMIT does not.

Fact Recall and Consistency

Fact recall is measured by the relatedness and consistency of generated facts in CoT responses. QLoRA maintains reasonable relatedness but low consistency, indicating memorization without generalization. MEMIT further degrades both metrics, especially in complex reasoning schemes.

Logical Coherence

Coherence of CoT responses is critical for reasoning. QLoRA-edited models show a slight decrease in coherence, while MEMIT-edited models suffer catastrophic loss of coherence, undermining their fundamental language modeling capabilities.

Discussion

QLoRA vs. MEMIT

QLoRA supports effective fact-wise editing and preserves logical coherence but fails to generalize edited knowledge for retrieval. MEMIT is inadequate for real-world factual knowledge, especially with complex subjects and relations, challenging the notion that edited neurons function solely as fact storage.

Model Scaling

Model scaling improves baseline knowledge and input-augmentation performance but does not enhance editing efficacy in QLoRA or MEMIT. Larger models do not exhibit increased factual effectiveness, fact retrieval, or coherence post-editing.

Implications and Future Directions

The findings highlight significant limitations in current knowledge editing methods, particularly in propagating updates for reasoning tasks. The inability to generalize and coherently reason with edited knowledge restricts practical deployment in dynamic knowledge environments. Future research should focus on:

• Enhancing generalization and retrieval of edited knowledge.
• Developing editing methods robust to complex, OpenIE-style facts.
• Integrating richer context during finetuning to improve recall.
• Addressing catastrophic forgetting in locate-and-edit approaches.

ReCoE provides a challenging benchmark for advancing knowledge editing research, emphasizing the need for methods that support coherent reasoning and robust propagation of updates.

Conclusion

This work introduces ReCoE, a reasoning-based benchmark for evaluating knowledge editing in LLMs, and demonstrates that existing methods fail to propagate updates for coherent reasoning. The analysis reveals critical deficiencies in fact recall and generation coherence, especially for locate-and-edit approaches. These insights establish a foundation for future research aimed at developing more effective and reliable knowledge editing techniques for LLMs.