Modeling the Severity of Complaints in Social Media (2103.12428v1)

Abstract: The speech act of complaining is used by humans to communicate a negative mismatch between reality and expectations as a reaction to an unfavorable situation. Linguistic theory of pragmatics categorizes complaints into various severity levels based on the face-threat that the complainer is willing to undertake. This is particularly useful for understanding the intent of complainers and how humans develop suitable apology strategies. In this paper, we study the severity level of complaints for the first time in computational linguistics. To facilitate this, we enrich a publicly available data set of complaints with four severity categories and train different transformer-based networks combined with linguistic information achieving 55.7 macro F1. We also jointly model binary complaint classification and complaint severity in a multi-task setting achieving new state-of-the-art results on binary complaint detection reaching up to 88.2 macro F1. Finally, we present a qualitative analysis of the behavior of our models in predicting complaint severity levels.

• The paper introduces a novel dataset that categorizes social media complaints into four severity levels, enabling precise face-threat analysis.
• It employs transformer-based networks enhanced with linguistic insights to achieve a macro F1 score of 55.7 for severity assessment.
• The multi-task learning framework significantly improves binary complaint detection, attaining a macro F1 score of 88.2 and guiding automated responses.

The paper "Modeling the Severity of Complaints in Social Media" explores the nuanced task of assessing the severity of complaints expressed in social media posts. This paper is rooted in the linguistic theory of pragmatics, which categorizes complaints by the degree of "face-threat" they pose, reflecting the intensity of dissatisfaction expressed. Understanding these severity levels is vital for gauging the intent behind complaints and can inform appropriate response strategies, particularly in customer service contexts.

The authors introduce an enriched dataset that augments a publicly available collection of complaints by categorizing them into four distinct severity levels. This provides a novel avenue for investigating complaint severity in computational linguistics, which had not been explored prior to this paper.

To address the task, the paper employs transformer-based networks—a state-of-the-art approach in natural language processing—enhanced with linguistic information to effectively model complaint severity. Their model achieves a macro F1 score of 55.7, indicating a balanced performance across the different severity classes.

Furthermore, the paper describes a multi-task learning approach, whereby both binary complaint classification and severity assessment are modeled simultaneously. This approach sets a new benchmark in binary complaint detection, achieving a macro F1 score of 88.2. The multi-task setup leverages shared representations to improve performance on both tasks.

Additionally, the paper includes a qualitative analysis of model predictions, offering insights into how different severity levels are interpreted by the models. This analysis helps in understanding the capabilities and limitations of the models in a real-world context.

In summary, this work contributes a novel dataset and demonstrates advanced methods in modeling complaint severity, paving the way for further research in sentiment analysis and automated customer service systems.