Question Suggestion for Conversational Shopping Assistants Using Product Metadata

Abstract: Digital assistants have become ubiquitous in e-commerce applications, following the recent advancements in Information Retrieval (IR), NLP and Generative AI. However, customers are often unsure or unaware of how to effectively converse with these assistants to meet their shopping needs. In this work, we emphasize the importance of providing customers a fast, easy to use, and natural way to interact with conversational shopping assistants. We propose a framework that employs LLMs to automatically generate contextual, useful, answerable, fluent and diverse questions about products, via in-context learning and supervised fine-tuning. Recommending these questions to customers as helpful suggestions or hints to both start and continue a conversation can result in a smoother and faster shopping experience with reduced conversation overhead and friction. We perform extensive offline evaluations, and discuss in detail about potential customer impact, and the type, length and latency of our generated product questions if incorporated into a real-world shopping assistant.

• The paper introduces a framework that combines in-context learning and supervised fine-tuning to automatically generate product-related questions.
• By using product metadata and buyer reviews, the method enhances question relevance, fluency, and diversity for smoother customer interactions.
• Evaluation through GPT-4 and human assessment confirms improved customer experience and efficient, real-time question generation.

Question Suggestion for Conversational Shopping Assistants Using Product Metadata

Introduction

The paper "Question Suggestion for Conversational Shopping Assistants Using Product Metadata" introduces a framework for enhancing the conversational capabilities of shopping assistants. By leveraging advancements in Information Retrieval (IR), NLP, and LLMs, the paper proposes using product metadata to generate useful, answerable, and diverse questions to guide customer interactions. This approach addresses shoppers' uncertainty and enhances the efficiency of shopping assistants by reducing friction in the conversational flow.

Framework and Methodology

The proposed system leverages LLMs to automatically generate product-related questions through two primary techniques: In-Context Learning (ICL) and Supervised Fine-Tuning (SFT). The goal is to produce questions grounded in the product's metadata and buyer reviews that are relevant, useful, answerable, fluent, styled appropriately for customer inquiries, and diverse.

• In-Context Learning: This method involves designing a prompt template that utilizes the LLM’s pre-existing knowledge to generate questions that meet predetermined criteria without explicit training examples.
• Supervised Fine-Tuning: This involves fine-tuning a large model with a curated dataset of product contexts and corresponding questions to enhance the generation quality. The dataset is created with minimal human intervention by initially employing the ICL approach and refining through manual inspection.

Evaluation

The generated questions are assessed both automatically using GPT-4 and through human evaluation. Key metrics include relevance, usefulness, answerability, fluency, and stylistic appropriateness. Both ICL and SFT approaches demonstrated strong performance across these metrics, with particular strengths in relevance and fluency. Differences in performance between the zero-shot, few-shot ICL, and SFT variants were noted, with the few-shot ICL sometimes outperforming SFT in producing stylistically appropriate questions.

Practical Implications

Incorporating such a question suggestion framework into real-world shopping assistants has potential benefits, such as:

• Improved Customer Experience: By providing pre-formulated questions, customers can navigate product information more efficiently, potentially increasing engagement and satisfaction.
• Efficient Conversations: With both broad and specific questions integrated into the shopping experience, customers have fewer obstacles in finding relevant information without typing extensive queries.
• Reduced Latency: Techniques like caching and streaming can be employed to handle real-time question generation without significant delays.

Conclusion

The research demonstrates a feasible method for enhancing conversational interactions with shopping assistants. By generating diverse and contextually grounded questions, this framework contributes to more intuitive and effective e-commerce experiences. Future directions involve integrating conversational history for personalized question generation and leveraging real-time feedback loops to refine the question generation process.

Further exploration in using instruction fine-tuning and incorporating customer behavior analytics promises additional refinements, driving more adaptive and customer-friendly shopping assistants.