KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning (1909.02151v1)

Abstract: Commonsense reasoning aims to empower machines with the human ability to make presumptions about ordinary situations in our daily life. In this paper, we propose a textual inference framework for answering commonsense questions, which effectively utilizes external, structured commonsense knowledge graphs to perform explainable inferences. The framework first grounds a question-answer pair from the semantic space to the knowledge-based symbolic space as a schema graph, a related sub-graph of external knowledge graphs. It represents schema graphs with a novel knowledge-aware graph network module named KagNet, and finally scores answers with graph representations. Our model is based on graph convolutional networks and LSTMs, with a hierarchical path-based attention mechanism. The intermediate attention scores make it transparent and interpretable, which thus produce trustworthy inferences. Using ConceptNet as the only external resource for Bert-based models, we achieved state-of-the-art performance on the CommonsenseQA, a large-scale dataset for commonsense reasoning.

Knowledge-Aware Graph Networks for Commonsense Reasoning

The paper "KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning" presents a sophisticated approach to enhancing machine commonsense reasoning through the use of external structured knowledge bases. This method leverages knowledge graphs to perform explainable and interpretable inferences, addressing one of the critical bottlenecks in the pursuit of artificial general intelligence.

Summary of Contributions

The research introduces KagNet, a novel framework designed to empower machines with the ability to perform commonsense reasoning by utilizing external, structured commonsense knowledge graphs, specifically ConceptNet. The framework functions by grounding a question-answer pair into a schema graph, which is a subgraph of the knowledge graph relevant to the given query. The novelty lies in the representation of these schema graphs via KagNet, which employs a combination of graph convolutional networks (GCNs), long short-term memory networks (LSTMs), and a hierarchical path-based attention (HPA) mechanism to produce transparent and trustworthy inferences.

Technical Approach

KagNet's architecture consists of several key components:

1. Schema Graph Grounding: The process begins by recognizing concepts in the question and answers, constructing schema graphs by finding paths among these concepts within ConceptNet, and then pruning noisy paths using knowledge graph embedding techniques. This ensures that the model focuses on the most relevant knowledge for reasoning.
2. Graph Representation via GCNs: The GCNs contextualize concept vectors within the schema graph, accommodating pre-trained embeddings in their particular context. This step refines the concept vectors and captures the structural patterns of the graphs.
3. Path Encoding with LSTMs: Path representations are derived from sequences of triples, capturing multi-hop relational information. This relational encoding is crucial for understanding implicit connections that facilitate commonsense reasoning.
4. Hierarchical Path-Based Attention: To address the issue of irrelevant paths, this attention mechanism selectively aggregates important path vectors and more critical question-answer concept pairs. This two-level attention structure allows the model to prioritize paths that contribute more significantly to reasoning.

Experimental Results

KagNet demonstrates state-of-the-art performance on the CommonsenseQA dataset, outperforming conventional fine-tuning methods of large pre-trained LLMs. It achieves a notable improvement in accuracy, showcasing the effectiveness of integrating structured external knowledge into commonsense reasoning tasks.

Importantly, the paper highlights the extensibility of the knowledge-centric approach, as KagNet shows superior transferability to other commonsense datasets like SWAG and Winograd Schema Challenge when compared to models that rely solely on linguistic pretraining. The inclusion of explicit, interpretable paths in reasoning not only elevates accuracy but also enhances transparency and user trust in AI systems.

Implications and Future Directions

The proposed KagNet framework offers significant implications for both the theoretical and practical aspects of AI research. Theoretically, it paves the way for more structured knowledge integration into neural architectures, which could improve reasoning capabilities beyond commonsense tasks. Practically, the successful application to commonsense question-answering settings indicates potential usage in areas requiring human-like inference capabilities, such as robotics and interactive systems.

Looking forward, the paper suggests future research directions, including refining question parsing to address negation, improving comparative reasoning, and extending the framework for multimodal problems like visual commonsense reasoning. These developments are essential for achieving more robust and generalizable AI systems capable of reasoning through diverse, real-world scenarios.