Generative AI at Work (2304.11771v2)

Abstract: We study the staggered introduction of a generative AI-based conversational assistant using data from 5,172 customer support agents. Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15\% on average, with substantial heterogeneity across workers. Less experienced and lower-skilled workers improve both the speed and quality of their output while the most experienced and highest-skilled workers see small gains in speed and small declines in quality. We also find evidence that AI assistance facilitates worker learning and improves English fluency, particularly among international agents. While AI systems improve with more training data, we find that the gains from AI adoption are largest for relatively rare problems, where human agents have less baseline training and experience. Finally, we provide evidence that AI assistance improves the experience of work along two key dimensions: customers are more polite and less likely to ask to speak to a manager.

• The paper demonstrates a 15% increase in resolutions per hour, particularly benefiting less experienced customer service agents.
• The paper analyzes 3 million chat records to assess handle time, resolution rates, and customer satisfaction as key productivity metrics.
• The paper finds a 38% adherence rate to AI recommendations, indicating that diligent use of AI accelerates agent learning and narrows skill gaps.

Generative AI at Work: A Study on Productivity and Work Experience in the Customer Service Industry

The paper "Generative AI at Work" presents an empirical analysis of the implementation of generative AI in the form of conversational assistants in a large-scale customer support setting. Using data from 5,172 customer service agents, the authors assess the impacts of AI-driven assistance on worker productivity, worker learning, and the overall experience of work. This paper provides valuable insights into the role of AI technologies in workplaces, specifically within the customer service sector, which is known for its high adoption rates of AI.

Overview of Findings

The authors employ a comprehensive dataset encompassing 3 million chat records to explore the effects of AI deployment on key productivity metrics: resolutions per hour, average handle time, chats handled per hour, resolution rates, and customer satisfaction. Their analysis reveals that AI assistance results in a 15% increase in resolutions per hour, with less experienced and lower-skilled workers experiencing the most pronounced gains. Importantly, AI assistance helps newer agents ascend the learning curve faster, reducing their ramp-up time significantly. Interestingly, while less experienced agents show marked improvements, more experienced agents exhibit smaller gains, suggesting differential impacts of AI assistance across skill levels.

Another pivotal finding concerns the dynamics of AI recommendation adherence. Agents exhibit a 38% adherence rate to AI suggestions, with those adopting recommendations more diligently reaping greater productivity benefits. Over time, adherence increases, particularly among initially skeptical or more experienced workers. This adherence pattern indicates a learning effect facilitated by AI advice, further corroborated by the observation that workers perform better even during system outages if they have had prior exposure to AI assistance.

Theoretical and Practical Implications

Theoretically, this paper challenges the conventional narrative of skill-biased technological change, suggesting that generative AI may narrow the productivity gap between lower and higher-skilled workers within the same occupation by disseminating best practices. This raises new questions about the interplay between AI tools and tacit knowledge, as these systems provide not only performance support but also act as vehicles for skill transference and tacit knowledge dissemination.

Practically, the deployment of AI in customer service settings shows potential for transformative impacts. Organizational strategies could be refined to focus on high-impact AI training for less experienced workers, while also addressing potential declines in quality among top performers due to over-reliance on AI recommendations. These findings also suggest that AI can play a significant role in improving the overall employee experience by reducing negative customer interactions and enhancing communication skills, particularly for international agents.

Future Developments and Speculations

Looking forward, the implications of this paper suggest several avenues for future research and policy development in the field of AI in the workplace. The long-term effects of AI on job design, employee satisfaction, and labor demand require further investigation. Future studies might also explore the macroeconomic effects of AI adoption across different sectors and occupations, focusing on how AI-facilitated learning can influence the trajectory of workforce development.

Moreover, questions regarding the compensation of workers who contribute to AI training datasets are becoming increasingly relevant. As AI systems evolve and potentially reshape the landscape of skill acquisition and deployment, it will be crucial for organizations and policymakers to develop frameworks that balance innovation with fair compensation and job satisfaction.

In conclusion, the paper "Generative AI at Work" provides a detailed examination of the first large-scale deployment of generative AI in a workplace setting, offering insights into the nuanced impacts of AI technologies on productivity and work environment. It contributes to our understanding of AI's role in modern work settings and paves the way for further exploration into the socio-economic implications of such technologies.