Commonsense for Generative Multi-Hop Question Answering Tasks (1809.06309v3)

Abstract: Reading comprehension QA tasks have seen a recent surge in popularity, yet most works have focused on fact-finding extractive QA. We instead focus on a more challenging multi-hop generative task (NarrativeQA), which requires the model to reason, gather, and synthesize disjoint pieces of information within the context to generate an answer. This type of multi-step reasoning also often requires understanding implicit relations, which humans resolve via external, background commonsense knowledge. We first present a strong generative baseline that uses a multi-attention mechanism to perform multiple hops of reasoning and a pointer-generator decoder to synthesize the answer. This model performs substantially better than previous generative models, and is competitive with current state-of-the-art span prediction models. We next introduce a novel system for selecting grounded multi-hop relational commonsense information from ConceptNet via a pointwise mutual information and term-frequency based scoring function. Finally, we effectively use this extracted commonsense information to fill in gaps of reasoning between context hops, using a selectively-gated attention mechanism. This boosts the model's performance significantly (also verified via human evaluation), establishing a new state-of-the-art for the task. We also show promising initial results of the generalizability of our background knowledge enhancements by demonstrating some improvement on QAngaroo-WikiHop, another multi-hop reasoning dataset.

Overview of "Commonsense for Generative Multi-Hop Question Answering Tasks"

The presented paper addresses the field of machine reading comprehension (MRC) within the context of generative question answering (QA) tasks, specifically focusing on the challenges posed by multi-hop reasoning requirements. Unlike traditional extractive QA tasks, which concentrate on directly retrieving factual information from texts, this research is situated in the complex domain of NarrativeQA—wherein a comprehensive grasp of the narrative structure and implicit relationships is imperative.

The authors propose a novel generative baseline: the Multi-Hop Pointer-Generator Model (MHPGM). This model leverages multiple attention mechanisms for iterative reasoning across multiple hops, coupled with a pointer-generator decoder for answer synthesis. The MHPGM demonstrates significant improvements over existing generative models and even holds competitive performance against state-of-the-art extractive models. Noteworthy are the scores of 41.49 for Rouge-L and 17.33 for METEOR on the NarrativeQA, indicative of its efficacy.

Building upon this, the research introduces an innovative approach for weaving in commonsense knowledge from ConceptNet, a semantic network, via a pointwise mutual information (PMI) and term-frequency-based selection algorithm. This commonsense information fills reasoning gaps through a specifically designed mechanism, the Necessary and Optional Information Cell (NOIC), which incorporates selectively gated attention. Experimental results indicate a substantial performance boost: the model now achieves 44.16 Rouge-L and 19.03 METEOR, establishing a new benchmark for the NarrativeQA dataset. Preliminary assessments on QAngaroo-WikiHop, another multi-hop reasoning dataset, suggest this commonsense-informed approach holds promise for broader applicability.

Implications and Future Directions

The implications of this research are two-fold: it offers practical advancements for AI models tackling MRC tasks and contributes to the theoretical understanding of incorporating external knowledge bases into neural architectures. The practicality of employing commonsense pathways to bridge cognitive gaps in reading comprehension tasks paves a pathway for more sophisticated NLP models capable of abstractive reasoning, enhancing capabilities beyond simple fact extraction.

Theoretical explorations inherent in the paper illustrate a fusion model that dynamically integrates structured knowledge (from ConceptNet) into generative reasoning frameworks. This indicates potential for future methods to expand capabilities into knowledge domains less explored by current datasets. As AI systems strive for increased human-like understanding, the development and application of such techniques to wider datasets—potentially incorporating diverse knowledge networks—seems a compelling trajectory.

Future research could extend beyond the single-domain commonsense network used here, potentially examining multi-domain inference where knowledge might be sourced from domain-specific ontologies or dynamically updated knowledge graphs. Additionally, real-world applications can leverage such integrative models in settings where inferential understanding is crucial, such as complex dialogue systems, narrative understanding for entertainment AI, and more.

In conclusion, the paper charts an intricate journey through MRC and commonsense incorporation within generative QA frameworks, providing substantial quantitative support and practical results that may inform both academic inquiries and operational AI system advancements in the foreseeable future.