Fuzzy Classification of Multi-intent Utterances

Abstract: Current intent classification approaches assign binary intent class memberships to natural language utterances while disregarding the inherent vagueness in language and the corresponding vagueness in intent class boundaries. In this work, we propose a scheme to address the ambiguity in single-intent as well as multi-intent natural language utterances by creating degree memberships over fuzzified intent classes. To our knowledge, this is the first work to address and quantify the impact of the fuzzy nature of natural language utterances over intent category memberships. Additionally, our approach overcomes the sparsity of multi-intent utterance data to train classification models by using a small database of single intent utterances to generate class memberships over multi-intent utterances. We evaluate our approach over two task-oriented dialog datasets, across different fuzzy membership generation techniques and approximate string similarity measures. Our results reveal the impact of lexical overlap between utterances of different intents, and the underlying data distributions, on the fuzzification of intent memberships. Moreover, we evaluate the accuracy of our approach by comparing the defuzzified memberships to their binary counterparts, across different combinations of membership functions and string similarity measures.

• The paper introduces a fuzzy membership model that assigns degrees of intent, capturing the inherent ambiguity of natural language.
• It employs an LSTM framework generating softmax scores, which are transformed via sigmoid and Gaussian functions to yield precise fuzzy memberships.
• Experiments on ATIS and SNIPS demonstrate that aggregating fuzzy memberships from single-intent data robustly enhances multi-intent recognition.

Fuzzy Classification of Multi-intent Utterances

Introduction

In contemporary dialog-based systems, the determination of user intents from utterances is predominantly executed using binary classification paradigms, which rely on crisp linguistic boundaries that fail to capture the intrinsic ambiguity of natural language. The research presented in "Fuzzy Classification of Multi-intent Utterances" addresses this inherent vagueness by proposing an approach that employs fuzzy logic principles to assign degrees of membership to intents, thus more accurately reflecting the spectrum of intent strengths contained within an utterance.

Fuzzy Membership and its Foundations

The proposed approach extends the traditional binary intent classification to a fuzzy membership model, which assigns utterances to intent classes based on degrees rather than absolutes. This model is particularly crucial in multi-intent scenarios where an utterance may simultaneously imply several intents to varying degrees. The research introduces both knowledge-based and data-driven methodologies for generating the parameters of fuzzy membership functions, thus reflecting both interpretability and adaptability in the mapping of softmax scores to linguistic fuzzy sets such as low, medium, and high.

Figure 1: Knowledge-based Membership.

Figure 2: Data-driven Membership.

Neural Networks and Softmax Scores

The implementation utilizes an LSTM-based neural network framework to generate softmax scores, which provide a probabilistic representation of class membership across various intents. These softmax outputs, representing likelihoods of intent categories, are interpreted through fuzzy logic to provide nuanced membership degrees. Sigmoid and Gaussian functions are employed to transform these probabilistic scores into meaningful fuzzy membership values across multiple intent classes, switch systematically classify the degree of association from low to high.

Multi-Intent Utterance Classification

Traditional systems for multi-intent recognition depend on extensive datasets of annotated multi-intent utterances, posing significant limitations due to the scarcity of such data. The methodology introduced in this work circumvents this limitation by utilizing data from single-intent utterances to inform multi-intent classification. By treating multi-intent processing as an approximate string matching problem, the study leverages fuzzy information retrieval (IR) techniques, including Jaccard similarity, cosine similarity, and other metrics to map complex multi-intent utterances onto known single-intent structures.

Figure 3: Fuzzy Membership Aggregation Module.

Fuzzy Membership Aggregation

For multi-intent utterances, the paper articulates a mechanism for aggregating multiple fuzzy membership degrees into a coherent singular representation. This process integrates the outcomes from multiple single-intent utterance matches, optimizing intent recognition through a similarity-weighted approach. Subsequently, these memberships are defuzzified into actionable insights using a carefully constructed piecewise function that crisply delineates intent classes based on membership thresholds.

Figure 4: Overall Architecture.

Experimental Evaluation

The study's experimental framework compares knowledge-based and data-driven strategies across ATIS and SNIPS datasets, demonstrating the superiority of adaptive, data-driven fuzzy membership generation. The evaluation highlights how lexical overlap impacts classification accuracy, revealing that fuzzy IR-based multi-intent recognition is particularly robust when utilizing token set ratio measures for matching. The results underscore the criticality of accounting for both linguistic overlap and inherent dataset distribution characteristics in fuzzy classification tasks.

Figure 5: Fuzzy intent memberships for multi-intent utterance.

Conclusion

The research establishes a novel paradigm for intent classification that accounts for the inherent vagueness and fuzzy nature of human language, which binary classification fails to address. Through leveraging single-intent utterance databases and fuzzy logic, the approach robustly classifies unseen multi-intent utterances without necessitating extensive multi-intent data. The results indicate promising avenues for enhancing dialog systems' natural language understanding capabilities, especially in environments where linguistic ambiguity is prevalent. Future works could focus on refining the adaptability and scalability of fuzzy membership models, particularly in multi-domain applications.