Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases (1903.02188v3)

Abstract: When answering natural language questions over knowledge bases (KBs), different question components and KB aspects play different roles. However, most existing embedding-based methods for knowledge base question answering (KBQA) ignore the subtle inter-relationships between the question and the KB (e.g., entity types, relation paths and context). In this work, we propose to directly model the two-way flow of interactions between the questions and the KB via a novel Bidirectional Attentive Memory Network, called BAMnet. Requiring no external resources and only very few hand-crafted features, on the WebQuestions benchmark, our method significantly outperforms existing information-retrieval based methods, and remains competitive with (hand-crafted) semantic parsing based methods. Also, since we use attention mechanisms, our method offers better interpretability compared to other baselines.

Analysis of Bidirectional Attentive Memory Networks for KBQA

The paper "Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases" by Chen, Wu, and Zaki addresses a prominent challenge in knowledge base question answering (KBQA): the lexical gap between natural language questions and the structured lexicon typical of knowledge bases (KBs). Existing methods generally fall into two categories: semantic parsing (SP) and information retrieval (IR)-based approaches. This work introduces a novel approach, the Bidirectional Attentive Memory Network (BAMnet), which surpasses existing IR-based methods and is competitive with some SP-based methods without reliance on extensive hand-crafted features or external resources.

Key Contributions

1. Bidirectional Attentive Memory Networks (BAMnet): BAMnet introduces a hierarchical two-way attention mechanism that models interactions between natural language questions and KBs. This bi-directional attention enables the system to capture important parts of both the question and relevant KB aspects, facilitating more accurate question answering.
2. Interpretability of Attention Models: The use of attention mechanisms provides interpretability to BAMnet, allowing it to focus on question components and KB details crucial for generating correct answers.
3. Performance Metrics: On the WebQuestions benchmark, BAMnet achieves a macro F1 score of 0.518 when using a topic entity predictor, notably surpassing other IR-based models. It achieves a superior score of 0.557 when provided with gold-standard topic entities.

Experimental Setup and Results

BAMnet is evaluated against the state-of-the-art models using the Freebase KB and the WebQuestions dataset. The model uses GloVe vectors for word embeddings and adopts a strategy of delexicalizing question entities to improve performance. The evaluation metrics focus on macro F1 scores, revealing that BAMnet effectively outperforms previous IR-based approaches while staying competitive with SP-based models that typically require extensive hand-crafted rules or external datasets.

Implications and Future Directions

The bidirectional attention mechanism proposed in this work opens up new possibilities for improving the interpretability and accuracy of KBQA systems by effectively capturing the dynamic interactions between questions and knowledge bases. Future research could explore extending these methods to handle more complex constraints, such as those involving comparisons, ordinality, or aggregations, which present significant challenges in current approaches.

This paper contributes significantly to the landscape of natural language processing by offering a scalable and interpretable solution to KBQA with implications for improving the accessibility of knowledge bases through natural language interfaces. The success of BAMnet demonstrates the potential of attention mechanisms in transforming KBQA systems, suggesting future iterations could further bridge the gap between questions and structured knowledge bases, potentially expanding into areas requiring deeper semantic understanding and reasoning.