Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations

Abstract: Despite the remarkable abilities of LLMs to answer questions, they often display a considerable level of overconfidence even when the question does not have a definitive answer. To avoid providing hallucinated answers to these unknown questions, existing studies typically investigate approaches to refusing to answer these questions. In this work, we propose a novel and scalable self-alignment method to utilize the LLM itself to enhance its response-ability to different types of unknown questions, being capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions. Specifically, the Self-Align method first employ a two-stage class-aware self-augmentation approach to generate a large amount of unknown question-response data. Then we conduct disparity-driven self-curation to select qualified data for fine-tuning the LLM itself for aligning the responses to unknown questions as desired. Experimental results on two datasets across four types of unknown questions validate the superiority of the Self-Align method over existing baselines in terms of three types of task formulation.

• The paper introduces a self-alignment framework enabling LLMs to recognize and explain limitations when faced with unanswerable questions.
• It employs a two-stage process with class-aware self-augmentation and disparity-driven self-curation to refine generated responses.
• Experimental results show significant improvements in unknown question detection and response quality compared to existing baselines.

Self-Aligning LLMs for Unknown Questions

Introduction

The paper addresses the common issue of overconfidence in LLMs when faced with unanswerable or ill-posed questions. Typically, LLMs have a tendency to provide confident yet inaccurate responses, resulting in hallucinated content. Existing methods largely focus on employing sophisticated reasoning or knowledge-enhanced techniques to improve accuracy when definitive answers exist. However, these approaches are suboptimal when questions lack definitive answers, often failing to determine when a question is inherently unanswerable.

Self-Alignment Method

The authors propose a scalable self-alignment approach that endows LLMs with the ability to recognize and explain their limitations when dealing with unknown questions. This method involves a two-stage process of class-aware self-augmentation followed by disparity-driven self-curation to fine-tune the LLM responses.

Figure 1: The workflow of the Self-Aligned method.

Class-Aware Self-Augmentation

Initially, the method employs a base LLM to generate unknown question-response data. This synthesis involves leveraging small seed data of known-unknown question pairs to instruct the model to rewrite known questions into different types of unknown categories.

1. Guided Question Rewriting: The model uses a few-shot learning technique with seed data as examples to convert known questions into unknown/questionable formats that reflect various types of unanswerability.
2. Conditioned Response Generation: The LLM then generates responses conditioned on an understanding of the question's unanswerability, providing explanations rather than incomplete answers.

Disparity-Driven Self-Curation

To improve the quality of the augmented data, the self-curation step assesses the disparity between the generated unknown question-answer pairs and their original known question-answer counterparts. This process filters out noisy or low-quality samples, ensuring only high-quality data is used for further model training.

Experimental Validation

The self-alignment method was rigorously tested on datasets specially curated for unknown questions. Experimental results indicate that the proposed method surpasses existing baselines significantly. The improvements are evident in key tasks such as unknown question detection, classification, and open-ended response generation, as shown by F1 scores and human evaluation metrics.

(Figure 2 and Figure 3)

Figure 2: Effect of self-curation approaches.

Figure 3: Effect of iterative self-alignment.

Discussion

The work establishes that enhancing LLMs' capability to recognize and appropriately handle their knowledge limitations leads to more reliable AI systems. Iterative self-alignment was identified as a process that further refines the model's effectiveness over successive cycles, a finding that suggests room for continued performance enhancements.

The case studies demonstrate that the Self-Aligned method enables LLMs to not only identify when a question is unanswerable but also articulate reasonable and coherent explanations, which is an advancement over merely refusing to answer.

Figure 4: Case study. The left one is an ambiguous question, while the right one is an incorrect question, illustrating hallucinated content and helpful explanations.

Conclusion

The research expands the understanding of how LLMs can self-align to improve responses to unknown questions, offering a significant advancement in generating trustworthy and accurate AI responses. The implications extend to various applications in conversational AI and autonomous systems where recognizing unknowns is crucial. Future developments may focus on enhancing the self-curation processes and exploring applications in interactive systems that require real-time learning and adaptation.

Overall, the paper proposes a robust framework that can be leveraged for other domains dealing with uncertain or incomplete information scenarios, marking a step towards more intelligent and self-aware AI systems.