Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning (2210.12217v1)

Abstract: Our goal is a question-answering (QA) system that can show how its answers are implied by its own internal beliefs via a systematic chain of reasoning. Such a capability would allow better understanding of why a model produced the answer it did. Our approach is to recursively combine a trained backward-chaining model, capable of generating a set of premises entailing an answer hypothesis, with a verifier that checks that the model itself believes those premises (and the entailment itself) through self-querying. To our knowledge, this is the first system to generate multistep chains that are both faithful (the answer follows from the reasoning) and truthful (the chain reflects the system's own internal beliefs). In evaluation using two different datasets, users judge that a majority (70%+) of generated chains clearly show how an answer follows from a set of facts - substantially better than a high-performance baseline - while preserving answer accuracy. By materializing model beliefs that systematically support an answer, new opportunities arise for understanding the model's system of belief, and diagnosing and correcting its misunderstandings when an answer is wrong.

An Expert Analysis of "Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning"

The paper "Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning" presents a significant advancement in the field of question-answering (QA) systems by introducing a framework that not only answers questions but also delineates how the answer logically follows from the system's internal beliefs. This approach offers a novel methodology for enhancing understanding of machine-generated answers, thereby facilitating greater transparency in QA systems.

Methodological Approach

The cornerstone of this research is the Entailer system, which leverages a backward-chaining model combined with a verification mechanism. The backward-chaining model generates premises that entail an answer hypothesis, executing recursive backward chaining to craft multistep proofs. The verifier corroborates these premises, ensuring they align with the model's own beliefs through self-querying, ultimately selecting the most convincing proof. Consequently, this system generates reasoning chains that are both faithful and truthful.

The design of Entailer marks a departure from existing QA systems, emphasizing systematic reasoning over explanation generation. While existing models might produce explanations through few-shot prompting or chains of thought, these are not necessarily faithful, nor do they reflect internal model beliefs. In contrast, Entailer materializes latent model knowledge explicitly as reasoning chains, offering insights into the model's belief structure and reasoning process for each answer it provides.

Evaluation and Results

The evaluation of Entailer was conducted on two datasets, OBQA and QuaRTz, where users assessed the clarity and reliability of the reasoning chains. Entailer outperforming baseline models with over 70% of the users concurring that the generated chains clearly illustrated how an answer followed from a set of facts. This outcome indicates substantial improvement in user-perceived accuracy and support of the answers compared to existing high-performance QA systems.

Entailer’s methodology ensures that reasoning-based QA accuracy remains competitive with direct QA accuracy. This suggests that producing reasoning chains does not detract from the efficacy of the system, but rather provides the added advantage of enhancing interpretability. The highlight of the finding is the ability of Entailer to preserve answer accuracy while furnishing explicit, structured reasoning chains.

Data and Training

The underlying model of Entailer was trained using a combination of the EntailmentBank dataset, enriched with a crowdsourced dataset to include both positive and negative entailments. Such comprehensive training data enables the model to perform zero-shot inference across various datasets. The architecture leverages T5-11B for multi-angle modeling, a testament to the robustness and flexibility of the approach.

Implications and Future Directions

The implication of Entailer’s design extends beyond mere QA tasks. By materializing model beliefs into coherent reasoning chains, it opens avenues for diagnosing and correcting erroneous model beliefs. This capability lays the groundwork for teachable systems that can evolve through user interactions, such as providing feedback to correct specific model beliefs—a path towards interactive machine learning systems.

Future developments in AI can harness this approach to cultivate systems that not only reason and answer questions but can also adapt over time through interaction with users. Incorporating dynamic memory systems could further enhance a model’s ability to learn from historical corrections, thus fostering improvement in answering accuracy and model reliability.

In summary, Entailer represents a pivotal step toward developing machine learning systems that support their answers with explicit chains of reasoning, offering both practical applications in enhancing QA systems and theoretical implications for understanding and modeling belief systems within AI architectures.