2000 character limit reached
Overview of Dialogue Robot Competition 2023 (2401.03547v1)
Published 7 Jan 2024 in cs.RO and cs.HC
Abstract: We have held dialogue robot competitions in 2020 and 2022 to compare the performances of interactive robots using an android that closely resembles a human. In 2023, the third competition DRC2023 was held. The task of DRC2023 was designed to be more challenging than the previous travel agent dialogue tasks. Since anyone can now develop a dialogue system using LLMs, the participating teams are required to develop a system that effectively uses information about the situation on the spot (real-time information), which is not handled by ChatGPT and other systems. DRC2023 has two rounds, a preliminary round and the final round as well as the previous competitions. The preliminary round has held on Oct.27 -- Nov.20, 2023 at real travel agency stores. The final round will be held on December 23, 2023. This paper provides an overview of the task settings and evaluation method of DRC2023 and the preliminary round results.
- K. Inoue, P. Milhorat, D. Lala, T. Zhao, and T. Kawahara, “Talking with ERICA, an autonomous android,” in Proc. of the Annual Meeting of the Special Interest Group on Discourse and Dialogue, 2016, pp. 212–215.
- G. R. Collins, “Improving human–robot interactions in hospitality settings,” International Hospitality Review, vol. 34, no. 1, pp. 61–79, 2020.
- D. Ham, J. Lee, Y. Jang, and K. Kim, “End-to-end neural pipeline for goal-oriented dialogue systems using GPT-2,” in Proc. of of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 583–592.
- T. Minato, R. Higashinaka, K. Sakai, T. Funayama, H. Nishizaki, and T. Nagai, “Design of a competition specifically for spoken dialogue with a humanoid robot,” Advanced Robotics, vol. 37, no. 21, pp. 1349–1363, 2023.
- C. T. Ishi, C. Liu, H. Ishiguro, and N. Hagita, “Evaluation of formant based-lip motion generation in tele-operated humanoid robots,” in Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012.
- K. Sakai, T. Minato, C. T. Ishi, and H. Ishiguro, “Speech driven trunk motion generating system based on physical constraint,” in Proc. of International Symposium on Robot and Human Interactive Communication, 2016.
- S. Katada, S. Okada, and K. Komatani, “Effects of physiological signals in different types of multimodal sentiment estimation,” IEEE Transactions on Affective Computing, vol. 14, no. 3, pp. 2443–2457, 2023.
- K. Komatani and S. Okada, “Multimodal human-agent dialogue corpus with annotations at utterance and dialogue levels,” in Proc. of of International Conference on Affective Computing & Intelligent Interaction, 2021, pp. 1–8.
- A. Shehata, M. A. A. Aljazaerly, L. Gaher, X. Li, Y. Makihara, and Y. Yagi, “Online model-based gait age and gender estimation,” in Proc. of the 7th IEEE International Joint Conference on Biometrics, 2023.
- M. Inaba, Y. Chiba, R. Higashinaka, K. Komatani, Y. Miyao, and T. Nagai, “Collection and analysis of travel agency task dialogues with age-diverse speakers,” in Proc. of the Thirteenth Language Resources and Evaluation Conference, 2022, pp. 5759–5767.
- OpenAI, “Gpt-4 technical report,” 2023, arXiv:2303.08774.
- K. Shukuri, R. Ishigaki, J. Suzuki, T. Naganuma, T. Fujimoto, D. Kawakubo, M. Shuzo, and E. Maeda, “Meta-control of dialogue systems using large language models,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- R. Hirai, S. Iizuka, H. Iseno, A. Guo, J. Jiang, A. Ohashi, and R. Higashinaka, “Team flow at DRC2023: Building common ground and text-based turn-taking in a travel agent spoken dialogue system,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- R. Ohashi, S. Agatsuma, K. Tsubokura, and Y. Iribe, “Team irisapu project description for DRC2023,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- H. Hong, H. Kawano, T. Maekawa, N. Yoshimaru, T. Iio, and K. Hatano, “A summarized history-based dialogue system for amnesia-free prompt updates,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- T. Miyama and S. Okada, “Android dialogue system for customer service using prompt-based topic control and compliments generation,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- F. Ito, T. Iwamoto, and K. Shigematsu, “Dialogue assistant system using ChatGPT’s function calling,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- Y. Kubo, T. Yamashita, and M. Yamada, “Dialogue system of team NTT-EASE for DRC2023,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- R. Yanagimoto, Y. Kubo, M. Oshio, M. Nakano, K. Yamamoto, and K. Komatanii, “User-adaptive tourist information dialogue system with yes/no classifier and sentiment estimator,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- K. Yoshikawa, T. Yamazaki, M. Ohagi, T. Mizumoto, and K. Sato, “Developing interactive tourism planning: A dialogue robot system powered by a large language model,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- K. Hisada and Y. Miyazaki, “Results of the Tokuolab team in the preliminary round of the dialogue robot competition 2023,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- N. Yoshimaru, M. Okuma, T. Iio, and K. Hatano, “AsyncMLD: Asynchronous multi-llm framework for dialogue recommendation system,” in Proc. of the Dialogue Robot Competition 2023, 2023.
- H. Onozeki, Z. Qi, K. Akiyama, R. Asahara, T. Kaneko, and M. Inaba, “Enhancing consistency in multimodal dialogue system using LLM with dialogue scenario,” in Proc. of the Dialogue Robot Competition 2023, 2023.