Which questions should I answer? Salience Prediction of Inquisitive Questions (2404.10917v2)

Abstract: Inquisitive questions -- open-ended, curiosity-driven questions people ask as they read -- are an integral part of discourse processing (Kehler and Rohde, 2017; Onea, 2016) and comprehension (Prince, 2004). Recent work in NLP has taken advantage of question generation capabilities of LLMs to enhance a wide range of applications. But the space of inquisitive questions is vast: many questions can be evoked from a given context. So which of those should be prioritized to find answers? Linguistic theories, unfortunately, have not yet provided an answer to this question. This paper presents QSALIENCE, a salience predictor of inquisitive questions. QSALIENCE is instruction-tuned over our dataset of linguist-annotated salience scores of 1,766 (context, question) pairs. A question scores high on salience if answering it would greatly enhance the understanding of the text (Van Rooy, 2003). We show that highly salient questions are empirically more likely to be answered in the same article, bridging potential questions (Onea, 2016) with Questions Under Discussion (Roberts, 2012). We further validate our findings by showing that answering salient questions is an indicator of summarization quality in news.

• The paper introduces a novel salience prediction model that identifies inquisitive questions enhancing text comprehension.
• It leverages a dataset of 1,766 annotated questions from news and TED talks, utilizing GPT-4 and Flan-T5 with instruction-tuning to outperform benchmarks.
• Empirical results demonstrate a strong correlation between question salience and in-text responses, promising improved summarization and discourse processing.

Enhancing Context Understanding through Salient Inquisitive Question Prediction

Introduction to Inquisitive Question Prediction

In this detailed exploration, the focus concentrates on the characterization and prediction of salient inquisitive questions that enhance text comprehension. Previous approaches in NLP have generated inquisitive questions to fulfill various analytical needs, but often without prioritizing question relevance or utility. This paper introduces a novel approach of developing a salience prediction model tailored to identify inquisitive questions whose answers substantially enhance understanding of the text.

Theoretical Background and Prior Work

Inquisitive questions arise naturally as readers seek to satisfy their curiosity about text content. The broad variability in these questions raises the challenge of distinguishing the most relevant or "salient" questions. Linguistically, this ties in with concepts of potential questions and Questions Under Discussion (QUDs), which reflect naturally in discourse progression. The authors provide a comprehensive overview of previous models that, while efficacious in generating numerous valid questions, lack mechanisms to assess their relative importance or salience.

Data and Methodology

The core of this research revolves around creating and utilizing a dataset of 1,766 inquisitive questions sourced from English news articles and TED talks, annotated for salience based on their contextual utility. The model predicts the salience of questions using a novel scoring system that factors in the extent to which a question's answers enhance text comprehension. The paper presents an innovative approach to model training, leveraging linguist-annotated salience scores alongside instruction-tuning techniques applied on known LLMs like GPT-4 and Flan-T5, which significantly outperform traditional zero-shot or few-shot approaches.

Empirical Findings and Model Evaluation

The empirical analysis demonstrates a strong correlation between the predicted salience of questions and their likelihood of being answered within the same article, suggesting alignment between question salience and discourse progression. The model's effectiveness is further illustrated through detailed performance metrics, specifically focusing on its capability to predict salience more accurately than established LLM benchmarks.

Practical Applications and Future Implications

The practical implications of this work are evident in its application to enhancing summary quality in journalistic contexts. By focusing on salient questions, summarization becomes more targeted and informative, potentially improving reader engagement and satisfaction. Looking forward, the proposed salience prediction model holds promise for broad applicability across various domains where information extraction and contextual interpretation are critical.

Conclusion

This paper marks a significant step toward understanding and automating the identification of salient inquisitive questions in textual data. By bridging linguistic theory with practical NLP applications, it opens new avenues for research and development in discourse processing, question generation, and beyond.

In summary, this research advances the field by presenting a robust, validated approach to predicting question salience, grounded in linguistic theory and enhanced by modern AI techniques. Its contributions are poised to influence future explorations and applications in automatic question generation and contextual information retrieval.