Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering (1909.05311v2)

Abstract: Commonsense question answering aims to answer questions which require background knowledge that is not explicitly expressed in the question. The key challenge is how to obtain evidence from external knowledge and make predictions based on the evidence. Recent works either learn to generate evidence from human-annotated evidence which is expensive to collect, or extract evidence from either structured or unstructured knowledge bases which fails to take advantages of both sources. In this work, we propose to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence. Specifically, we extract evidence from both structured knowledge base (i.e. ConceptNet) and Wikipedia plain texts. We construct graphs for both sources to obtain the relational structures of evidence. Based on these graphs, we propose a graph-based approach consisting of a graph-based contextual word representation learning module and a graph-based inference module. The first module utilizes graph structural information to re-define the distance between words for learning better contextual word representations. The second module adopts graph convolutional network to encode neighbor information into the representations of nodes, and aggregates evidence with graph attention mechanism for predicting the final answer. Experimental results on CommonsenseQA dataset illustrate that our graph-based approach over both knowledge sources brings improvement over strong baselines. Our approach achieves the state-of-the-art accuracy (75.3%) on the CommonsenseQA leaderboard.

Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering

The paper entitled "Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering" presents a sophisticated approach to addressing the challenges inherent in commonsense question answering tasks. The principal focus of this research is leveraging external knowledge in both structured and unstructured forms to enhance commonsense reasoning. The authors introduce a methodology that extracts evidence from heterogeneous sources, specifically targeting structured data from ConceptNet and unstructured data from Wikipedia articles, and subsequently employs graph-based reasoning over the collated evidence.

Core Methodology

1. Knowledge Extraction: The paper elucidates methods for extracting pertinent knowledge from ConceptNet and Wikipedia. ConceptNet provides structured data in the form of relational triples, which are transformed into graph paths for analysis. Concurrently, sentences from Wikipedia are gathered using Elastic Search, which are then graphically structured using Semantic Role Labeling (SRL). This dual-source knowledge strategy aims to maximize the coverage and depth of commonsense knowledge available.
2. Graph Construction: Evidence from ConceptNet is organized into Concept-Graphs, structuring relational paths between entities. Similarly, from Wikipedia, triples are derived from sentences, forming the basis of the Wiki-Graphs. The approach fuses these graphs to create a comprehensive representation, allowing the model to incorporate both node-specific and relationship-specific data from the evidence.
3. Graph-Based Reasoning: The paper proposes two innovative modules for reasoning:
   • Graph-Based Contextual Representation Learning: Utilizing XLNet as a backbone, the researchers redefine word distances within texts based on graph structure, improving contextual word representation by employing a topology sort algorithm.
   • Graph-Based Inference Module: Graph Convolutional Networks (GCNs) are utilized to propagate node information efficiently, followed by a graph attention mechanism to aggregate evidence and derive final predictions.

Experimental Results

The presented approach was evaluated using the CommonsenseQA dataset, where the authors demonstrated that their method improves upon strong baseline models. Notably, the accuracy achieved was 75.3%, setting a new benchmark for state-of-the-art performance on this task.

Implications and Future Work

The implications of this research are multi-faceted, offering a robust framework for integrating heterogeneous external knowledge sources into AI reasoning tasks. The successful integration of structured and unstructured data into a coherent graph-based model could extend beyond commonsense QA to other domains requiring nuanced reasoning over diverse datasets, such as medical diagnosis or legal interpretation.

Moreover, the paper suggests potential future directions to expand the capabilities of AI systems in common sense tasks. Important avenues for development include refining evidence extraction strategies to encompass a broader range of datasets, enhancing natural language templates for better integration of structured data, and optimizing graph neural network architectures to handle more complex, large-scale graphs.

In summary, the paper contributes significantly to the field of commonsense reasoning in artificial intelligence by offering a novel approach to graph-based reasoning over heterogeneous knowledge sources. This work provides a foundation upon which more comprehensive and accurate AI reasoning systems can be built, highlighting the necessity of integrating diverse forms of external data to improve machine understanding and inference.