Contact Complexity in Customer Service (2402.15655v1)

Abstract: Customers who reach out for customer service support may face a range of issues that vary in complexity. Routing high-complexity contacts to junior agents can lead to multiple transfers or repeated contacts, while directing low-complexity contacts to senior agents can strain their capacity to assist customers who need professional help. To tackle this, a machine learning model that accurately predicts the complexity of customer issues is highly desirable. However, defining the complexity of a contact is a difficult task as it is a highly abstract concept. While consensus-based data annotation by experienced agents is a possible solution, it is time-consuming and costly. To overcome these challenges, we have developed a novel machine learning approach to define contact complexity. Instead of relying on human annotation, we trained an AI expert model to mimic the behavior of agents and evaluate each contact's complexity based on how the AI expert responds. If the AI expert is uncertain or lacks the skills to comprehend the contact transcript, it is considered a high-complexity contact. Our method has proven to be reliable, scalable, and cost-effective based on the collected data.

• The paper introduces a machine learning model that defines customer contact complexity without human consensus by emulating senior agent decisions.
• The model, leveraging a GBDT with TF-IDF on 450,000 transcripts, achieved an impressive 0.96 AUC and 0.93 top-15 accuracy in predicting 152 issue codes.
• The study integrates complexity attributes—transcript length, uncertainty, and skillfulness—into a unified score that optimizes contact routing and resource allocation.

The paper, "Contact Complexity in Customer Service," presents a novel approach to addressing the challenges of efficiently routing customer service contacts based on their complexity. The authors identify the traditional limitations of routing systems that rely on product line or service type, as these do not capture the variable complexity inherent in customer issues. Instead, they propose a machine learning model to evaluate contact complexity, which can enhance the routing process by directing contacts to the appropriate level of customer service agent and thereby improve customer satisfaction and optimize resource allocation.

Key Contributions and Methodology

1. Complexity Definition Without Human Annotation: The paper introduces a method to define contact complexity without relying on costly and time-consuming consensus-based annotation by experienced agents. Instead, the model employs an AI expert to emulate the decision process of senior agents, thereby labeling contact complexity based on the AI's ability to respond accurately to contact transcripts.
2. AI Expert Model: The model is trained on a dataset from Amazon's MessageUS system. It utilizes 450,000 contact transcripts and employs a gradient boosting decision tree (GBDT) model with TF-IDF text embedding to predict one of 152 standardized issue codes (SIC). The AI expert's skillfulness in mimicking the routing decision process is demonstrated by an AUC$_{\mu}$ score of 0.96 and a top-15 accuracy of 0.93.
3. Complexity Hypotheses Conversion: The authors quantify contact complexity using three attributes: length, uncertainty, and skillfulness. Length is measured by the number of agent sentences in a transcript; uncertainty is quantified by the entropy of the AI expert's predictions; and skillfulness is evaluated using the integral of a KL divergence boosting function over the tree index in the model.
4. Complexity Score: The separate components of complexity are transformed and combined to form a single complexity score, $\mathcal{C}$. The score is then further refined through quantile transformation to form a "relative complexity score," $\mathcal{Q}$, which ranges from 0 to 1, providing a simple and interpretable measure of contact complexity.
5. Validation: The paper validates the complexity score with both indirect and direct methods. Indirect validation shows low resolution rates and high transfer rates for contacts with high complexity scores. Direct validation involves senior agent reviews, showing that model-generated scores align well with human judgments of complexity.

Implications and Future Work

The proposed method allows for the efficient routing of customer service contacts by integrating a complexity-based model into the routing workflow. This integration can potentially reduce unnecessary transfers, decrease handling time, and optimize the use of both junior and senior agents. Furthermore, the method provides valuable features such as complexity scores and hypotheses for analysis and feature generation in subsequent machine learning models.

Looking forward, the authors suggest augmenting the AI expert with additional labels and utilizing ensemble models to further refine the measurement of contact complexity. This expansion promises robustness in complexity evaluation, enhancing the model's applicability across various real-world scenarios in the customer service domain. Such innovations contribute significantly to improving operational efficiency and customer satisfaction in large-scale e-commerce environments.