Deep Open Intent Classification with Adaptive Decision Boundary (2012.10209v5)

Abstract: Open intent classification is a challenging task in dialogue systems. On the one hand, it should ensure the quality of known intent identification. On the other hand, it needs to detect the open (unknown) intent without prior knowledge. Current models are limited in finding the appropriate decision boundary to balance the performances of both known intents and the open intent. In this paper, we propose a post-processing method to learn the adaptive decision boundary (ADB) for open intent classification. We first utilize the labeled known intent samples to pre-train the model. Then, we automatically learn the adaptive spherical decision boundary for each known class with the aid of well-trained features. Specifically, we propose a new loss function to balance both the empirical risk and the open space risk. Our method does not need open intent samples and is free from modifying the model architecture. Moreover, our approach is surprisingly insensitive with less labeled data and fewer known intents. Extensive experiments on three benchmark datasets show that our method yields significant improvements compared with the state-of-the-art methods. The codes are released at https://github.com/thuiar/Adaptive-Decision-Boundary.

• The paper introduces a novel method using an adaptive decision boundary and new loss function to better balance known and unknown intent classification in dialogue systems.
• The proposed method achieves superior performance on benchmark datasets and demonstrates robustness even with limited labeled data or fewer known intents.
• The adaptive decision boundaries learned by the model precisely adjust to the feature space of known intents, enhancing the ability to distinguish unknown intents.

Deep Open Intent Classification with Adaptive Decision Boundary

The paper "Deep Open Intent Classification with Adaptive Decision Boundary" addresses the challenge of open intent classification within dialogue systems. This task involves not only identifying known intents accurately but also detecting unknown intents without prior data on those unknown intents. The authors propose a novel methodology that seeks to resolve the limitations of existing models, which struggle with balancing the identification of known and unknown intents due to inappropriate decision boundaries.

The central contribution of the paper is a post-processing technique that allows the learning of an adaptive decision boundary (ADB) for open intent classification. The authors suggest first using labeled known intent samples to pre-train their model. Subsequently, they propose an automatic learning phase to determine adaptive spherical decision boundaries for each known category based on well-trained features. Crucially, they introduce a new loss function designed to balance empirical risk and open space risk, which is vital for the adaptive nature of the decision boundaries. This innovation eliminates the need for open intent samples and removes the necessity to alter the architecture of the model. Moreover, the method demonstrates insensitivity to reduced labeled data and decreased numbers of known intents, showcasing robust performance across various conditions.

Their methodological approach begins with extracting intent features via BERT, followed by initializing centroids and decision boundary parameters for known classes. The learning process involves a boundary loss function that adjusts these parameters to encircle known samples while mitigating the risk of misclassification. The effectiveness of this method is evidenced by significant improvements on three benchmark datasets: BANKING, OOS (Out-Of-Scope), and StackOverflow. Across various known class proportions, their model consistently achieves superior results in terms of accuracy and macro F1-score compared to state-of-the-art techniques such as OpenMax, DOC, and DeepUnk.

A thorough analysis presented in the work indicates that the learned decision boundaries are precisely adjusted to the feature space of known intents, leading to an improved balance between minimizing empirical risk and open space risk. This balance is critical for successfully distinguishing unknown intents from known intents, as demonstrated by the superior detection performance in empirical evaluations. One of the paper’s significant insights is the robustness of their model even when labeled data availability is scarce, a compelling feature that addresses practical concerns in real-world applications where labeled data is often limited.

From a theoretical perspective, this work advances the understanding of open world classification in dialogue systems by offering a highly adaptable and efficient method for intention detection. Practically, the findings hold potential for enhancing the development of dialogue systems by improving user satisfaction through more accurate intent recognition, which could lead to better user interactions in systems where the intent is a pivotal component.

The implications of this research extend to future work in AI systems, where uncertainty and new classes of data frequently arise, thus necessitating models that can autonomously adjust decision boundaries to accommodate such changes. Future developments might explore further generalizations of this approach across different domains and utilize automated boundary adaptation in more complex conversational models.

In conclusion, the paper makes a valuable contribution to the field of dialogue systems by providing a robust and scalable solution to the challenge of open intent classification, paving the way for more adaptive and intelligent systems in various AI applications.