Rapport-Driven Virtual Agent: Rapport Building Dialogue Strategy for Improving User Experience at First Meeting (2406.09839v1)

Abstract: Rapport is known as a conversational aspect focusing on relationship building, which influences outcomes in collaborative tasks. This study aims to establish human-agent rapport through small talk by using a rapport-building strategy. We implemented this strategy for the virtual agents based on dialogue strategies by prompting a LLM. In particular, we utilized two dialogue strategies-predefined sequence and free-form-to guide the dialogue generation framework. We conducted analyses based on human evaluations, examining correlations between total turn, utterance characters, rapport score, and user experience variables: naturalness, satisfaction, interest, engagement, and usability. We investigated correlations between rapport score and naturalness, satisfaction, engagement, and conversation flow. Our experimental results also indicated that using free-form to prompt the rapport-building strategy performed the best in subjective scores.

• The paper demonstrates that integrating rapport-building strategies, particularly through free-form dialogue, significantly improves user engagement in human-agent interactions.
• The study employs both predefined and adaptive dialogue methods, evaluating impact through metrics like rapport scores, naturalness, and satisfaction.
• Results reveal strong correlations between rapport and user experience factors, underscoring the potential for more human-like and effective virtual agents.

Rapport-Driven Virtual Agent: Rapport Building Dialogue Strategy for Improving User Experience at First Meeting

The paper "Rapport-Driven Virtual Agent: Rapport Building Dialogue Strategy for Improving User Experience at First Meeting" by Baihaqi, García Contreras, Kawano, and Yoshino presents a paper focusing on the importance of rapport in Human-Agent Interaction (HAI). The authors delve into enhancing user experience and the outcome of collaborative tasks through rapport-building strategies integrated into small talk conducted by virtual agents.

Introduction and Motivation

The research acknowledges the imperative role of rapport in fostering effective communication and a positive atmosphere in collaborative tasks. Unlike traditional task-oriented HAI systems, which concentrate mainly on the immediate outcomes, this paper posits that emphasizing relational aspects such as rapport can significantly improve overall user experiences. By integrating rapport-building dialogue strategies, particularly during first meetings, the paper aims to not only enhance user perceptions but also potentially improve future interactions and task outcomes.

Methodology

The authors implemented rapport-building strategies using predefined sequences and free-form dialogue methods in a virtual agent platform named ERICA. The virtual agent employed various rapport-building utterances derived from prior studies, such as participation appreciation, praise expression, self-disclosure, knowledge sharing, empathetic response, storytelling, recommendation giving, positive encouragement, joke sharing, and name usage.

The paper utilized a systematic approach by integrating these utterances either in a structured predefined manner or in a more spontaneous free-form style. The virtual agent's effectiveness was evaluated through human interactions, with key metrics including total dialogue turns, utterance characters, rapport score (RS), and various user experience (UX) variables: naturalness, satisfaction, interest, engagement, and usability.

Experimental Setup

The experiments involved 20 participants, all native Japanese speakers, who were engaged in small talk with various versions of the virtual agent. The four distinct types of agents evaluated were:

1. Limit Free Rapport Agent: No turn limit, free-form dialogue strategy.
2. Free Rapport Agent: Turn limit of 20, free-form dialogue strategy.
3. Predefined Rapport Agent: Predefined scenario, 20 turns.
4. Q&A Agent: Traditional question-answer format.

The interactions covered topics such as dream travel destinations, allowing exploration of both personal and universal themes.

Results

Correlation Analysis

The paper revealed substantial correlations between the rapport score (RS) and certain UX variables. Notably, RS exhibited moderate to strong correlations with naturalness (ρ = 0.525), satisfaction (ρ = 0.662), engagement (ρ = 0.706), and conversational flow (ρ = 0.512). These findings underscore the critical importance of rapport in enhancing user perception and interaction quality. Interestingly, the total turns and utterance characters showed no significant correlation with RS or UX variables, suggesting that qualitative aspects of dialogue play a more decisive role than quantitative measures.

Strategy Comparison

In comparing the predefined and free-form strategies, the Free Rapport Agent significantly outperformed the Predefined Rapport Agent across several UX dimensions, particularly in naturalness, satisfaction, and usability. This indicates that while structured approaches offer consistency, the adaptability of free-form dialogue resonates more positively with users, facilitating more engaging and natural interactions.

Effect of Rapport-Building Strategy

When comparing the Free Rapport Agent with the traditional Q&A Agent, the former demonstrated superior performance in terms of interest, engagement, and overall rapport. This highlights the added value of incorporating specific rapport-building elements into dialogue systems, which not only enhances the interaction quality but also fosters a sense of connection between the human and the virtual agent.

Discussion and Implications

The findings of this paper have significant implications for the development of more effective virtual agents. The absence of a correlation between total dialogue length and rapport underscores the necessity of focusing on the quality of interactions rather than merely increasing the quantity. Moreover, the success of the Free Rapport Agent exemplifies the potential of adaptive dialogue strategies in creating more meaningful and engaging user experiences.

Future research could further explore the integration of emotional TTS and non-verbal cues like nodding to enrich the interaction, potentially expanding the impact of positive reinforcement and empathetic responses. Such enhancements could deepen the sense of rapport and improve the overall effectiveness of virtual agents in various applications, including healthcare, education, and customer service.

Conclusion

The paper successfully demonstrates that rapport-building strategies significantly improve user experience in HAI. By leveraging free-form dialogue methods and targeted rapport-building utterances, virtual agents can foster more engaging, satisfying, and natural interactions. These findings pave the way for future advancements in the design and implementation of virtual agents, underscoring the integral role of rapport in enhancing human-agent relationships.

This essay provides a comprehensive summary of the paper, emphasizing key findings, methodologies, and implications for the future of virtual agents in various domains. The authors' research contributes valuable insights into the importance of rapport in HAI, offering promising directions for developing more effective and human-like virtual agents.