Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Proving (2405.01379v4)

Abstract: Natural language explanations represent a proxy for evaluating explanation-based and multi-step Natural Language Inference (NLI) models. However, assessing the validity of explanations for NLI is challenging as it typically involves the crowd-sourcing of apposite datasets, a process that is time-consuming and prone to logical errors. To address existing limitations, this paper investigates the verification and refinement of natural language explanations through the integration of LLMs and Theorem Provers (TPs). Specifically, we present a neuro-symbolic framework, named Explanation-Refiner, that integrates TPs with LLMs to generate and formalise explanatory sentences and suggest potential inference strategies for NLI. In turn, the TP is employed to provide formal guarantees on the logical validity of the explanations and to generate feedback for subsequent improvements. We demonstrate how Explanation-Refiner can be jointly used to evaluate explanatory reasoning, autoformalisation, and error correction mechanisms of state-of-the-art LLMs as well as to automatically enhance the quality of explanations of variable complexity in different domains.

Exploring Explanation-Refiner: A Neuro-Symbolic Approach to Refining NLI Explanations

Introduction to Explanation-Refiner

The process of generating natural language explanations alongside predictions is becoming increasingly significant in improving the transparency and understandability of AI models, particularly in Natural Language Inference (NLI) tasks. The recent integration of LLMs (LMs) with logical frameworks like Theorem Provers (TPs) has opened new avenues for enhancing the quality of these explanations. The Explanation-Refiner framework is a novel approach that taps into both LLMs and TPs to not only generate but also refine explanations for NLI. This combination allows for more rigorous validation and improvement of the explanations.

The Crux of Explanation-Refiner

Explanation-Refiner establishes a symbiotic relationship between LLMs and TPs. Here's how the framework operates:

• LLMs generate initial explanatory sentences from given texts.
• TPs verify these explanations against logical criteria.
• If inaccuracies or logical fallacies are found, the TPs provide detailed feedback about the mismatches or errors.
• LLMs then use this feedback to refine and correct the explanations.

Particularly notable is the use of state-of-the-art LLMs like GPT-4 and Isabelle/HOL as the theorem proving assistant, which enables high accuracy and detailed logical scrutiny.

Combating Challenges in Explanation Validation

The generation of natural language explanations, particularly in complex datasets, faces numerous challenges including incompleteness and susceptibility to error. Traditional metrics and crowd-sourcing methods used to validate these explanations often fall short, especially when it comes to capturing the subtleties required for robust logical reasoning. Explanation-Refiner addresses these limitations efficiently by leveraging the precision of formal logic checks via TPs.

Practical Implications

The implications of such a neuro-symbolic integration are profound:

• Improved Accuracy: By continuously refining explanations through logical validation, the accuracy of these explanations is significantly enhanced.
• Feedback Loop: The framework offers an iterative refinement process, where LLM-generated explanations are incrementally improved based on specific feedback from TPs.
• Scalability Across Domains: Initial experiments across different complexity levels and domains (e-SNLI, QASC, WorldTree) show promise for broader applicability.
• Syntax Error Reduction: By focusing on refining explanations at a syntactic level (68.67% average reduction in syntax errors), the framework also aids in enhancing the LLM’s output quality.

Future Directions and Speculations

While the current instance of Explanation-Refiner leds substantial improvements, the journey doesn’t end here. Further refinements can make the process more efficient, reducing the number of iterations needed for satisfactory explanation enhancement. Moreover, the adaptability of this framework could open up possibilities for its application in other areas of AI where explanation integrity is crucial. We might also explore how different configurations of LLMs and TPs can affect the refinement effectiveness, potentially leading to custom setups for specific types of NLI tasks.

In essence, Explanation-Refiner not only underscores the vital role of explainability in AI but also actively contributes to the evolution of more understandable and logically consistent NLI models. Looking forward, the continual development of such frameworks is likely to play a critical part in the crafting of trustworthy AI systems.