Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question Answering

Abstract: Open Domain Question Answering requires systems to retrieve external knowledge and perform multi-hop reasoning by composing knowledge spread over multiple sentences. In the recently introduced open domain question answering challenge datasets, QASC and OpenBookQA, we need to perform retrieval of facts and compose facts to correctly answer questions. In our work, we learn a semantic knowledge ranking model to re-rank knowledge retrieved through Lucene based information retrieval systems. We further propose a "knowledge fusion model" which leverages knowledge in BERT-based LLMs with externally retrieved knowledge and improves the knowledge understanding of the BERT-based LLMs. On both OpenBookQA and QASC datasets, the knowledge fusion model with semantically re-ranked knowledge outperforms previous attempts.