Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering (1911.10470v2)

Abstract: Answering questions that require multi-hop reasoning at web-scale necessitates retrieving multiple evidence documents, one of which often has little lexical or semantic relationship to the question. This paper introduces a new graph-based recurrent retrieval approach that learns to retrieve reasoning paths over the Wikipedia graph to answer multi-hop open-domain questions. Our retriever model trains a recurrent neural network that learns to sequentially retrieve evidence paragraphs in the reasoning path by conditioning on the previously retrieved documents. Our reader model ranks the reasoning paths and extracts the answer span included in the best reasoning path. Experimental results show state-of-the-art results in three open-domain QA datasets, showcasing the effectiveness and robustness of our method. Notably, our method achieves significant improvement in HotpotQA, outperforming the previous best model by more than 14 points.

Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering

The paper, "Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering," presents a novel approach to open-domain question answering (QA) that emphasizes multi-hop reasoning. This approach addresses the challenge of retrieving relevant evidence spanning multiple documents, which conventional single-hop QA systems struggle with, due to constraints on relational data retrieval between documents.

Methodological Advancements

The authors propose a method based on a graph-structured representation of Wikipedia, wherein nodes represent paragraphs and edges are either Wikipedia hyperlinks or within-document links. This graph formulation naturally supports multi-hop reasoning by facilitating path traversal between semantically related paragraphs.

1. Recurrent Graph-Based Retriever: This component tackles multi-hop retrieval by leveraging a recurrent neural network (RNN) architecture. The RNN sequentially selects paragraphs, conditioned on previously retrieved ones, constructing reasoning paths across the graph. It initializes with a TF-IDF retrieved set of paragraphs from Wikipedia, significantly reducing the search space.
2. Reader Model and Re-Ranking: To extract the final answer, a reading comprehension model is employed. This model re-ranks reasoning paths using contextual BERT embeddings, fine-tuned to capture relationships within the concatenated paragraphs of each path.

Experimental Insights

The approach was rigorously tested across three QA datasets: HotpotQA, SQuAD Open, and Natural Questions Open. The model achieved state-of-the-art results, notably in HotpotQA's full wiki setting, with impressive performance gains—particularly a more than 14-point improvement in F1 over previous models. The robustness of this method across datasets underscores its versatility in handling various open-domain QA challenges.

Practical and Theoretical Implications

The empirical results highlight several important aspects:

• Enhanced Reasoning Capabilities: By structuring Wikipedia as a graph and employing multi-hop retrieval, the method captures complex relationships between entities, reflecting a more human-like reasoning process. This aligns well with theoretical advances in representation learning that argue for contextual embeddings in multi-turn reasoning.
• Scalability and Efficiency: The initial use of TF-IDF to filter candidate paragraphs ensures scalability, an essential consideration for real-world applications dealing with vast information landscapes.

Future Directions

Considering the space of future developments, integrating end-to-end training for the graph-based retriever and reader could refine path selection and answer accuracy. Additionally, exploring graph structures beyond Wikipedia—extending to sources with heterogeneous link types—could further diversify the model's capabilities and resilience to varying data quality and structures.

In summary, this paper contributes a powerful method to the QA ecosystem, blending graph-based reasoning with deep learning to navigate complex informational terrain, fostering advancements in both model interpretability and performance.