Controllable Decontextualization of Yes/No Question and Answers into Factual Statements (2401.09775v1)

Abstract: Yes/No or polar questions represent one of the main linguistic question categories. They consist of a main interrogative clause, for which the answer is binary (assertion or negation). Polar questions and answers (PQA) represent a valuable knowledge resource present in many community and other curated QA sources, such as forums or e-commerce applications. Using answers to polar questions alone in other contexts is not trivial. Answers are contextualized, and presume that the interrogative question clause and any shared knowledge between the asker and answerer are provided. We address the problem of controllable rewriting of answers to polar questions into decontextualized and succinct factual statements. We propose a Transformer sequence to sequence model that utilizes soft-constraints to ensure controllable rewriting, such that the output statement is semantically equivalent to its PQA input. Evaluation on three separate PQA datasets as measured through automated and human evaluation metrics show that our proposed approach achieves the best performance when compared to existing baselines.

• The paper introduces a novel Transformer-based model that employs a soft-constraints mechanism to transform Yes/No question-answer pairs into standalone factual statements.
• The paper leverages automatically extracted constraints from QA constituency parse trees to ensure semantic equivalence and context independence.
• The paper demonstrates superior performance over baselines across multiple datasets with improved BLEU, ROUGE, and BertScore metrics alongside favorable human evaluations.

Overview

Understanding and extracting knowledge from Yes/No questions (PQA) across digital platforms such as forums and e-commerce sites is a complex task due to the contextual nature of such exchanges. This paper tackles the challenge of transforming answers to Yes/No questions into decontextualized factual statements. The authors present a novel Transformer-based sequence to sequence model with a mechanism for controllable rewriting using soft-constraints, aiming to maintain the semantic equivalence of the output to the original question-answer pairs.

Methodology

The proposed model operates by taking the polar question, its binary answer (yes or no), and any additional relevant context as inputs. The model then applies constraint-based rewriting to generate a stand-alone factual statement that encapsulizes the question and answer in an informative manner. This method primarily utilizes a soft-constraints mechanism to ensure the produced statements are not only semantically aligned with the original PQA but also appropriate for use out of their initial context - a crucial requirement for voice-assistants and related applications. The constraints are automatically extracted from the input PQA based on QA constituency parse trees, marking a significant departure from methods that rely on manual constraint specification.

Evaluation

The model's performance was evaluated against three separate PQA datasets and compared to existing baseline methods through both automated metrics and human evaluations. Automated metrics such as BLEU, ROUGE, and BertScore were used, alongside human evaluators who assessed statement syntactic clause coverage and overall coherence, correctness, and factualness. The proposed method outperformed the baselines, indicating that it more effectively generates accurate, coherent, and contextually independent statements from PQA inputs.

Potential Applications and Generalizability

The robustness of the approach becomes clear through further testing of zero-shot generalization across out-of-domain sets from Reddit and SemEval. The results suggest that the model isn't merely memorizing lexical content but is capable of genuine syntactic and semantic rewriting, which carries promising implications for diverse implementation contexts. By refining the conversion of polar question-answer pairs into factual statements, this work could significantly contribute to knowledge extraction and question-answering systems, laying the groundwork for more adaptive and versatile AI applications in the domain of natural language processing.