Modular Customizable ROS-Based Framework for Rapid Development of Social Robots (2311.15780v1)

Abstract: Developing socially competent robots requires tight integration of robotics, computer vision, speech processing, and web technologies. We present the Socially-interactive Robot Software platform (SROS), an open-source framework addressing this need through a modular layered architecture. SROS bridges the Robot Operating System (ROS) layer for mobility with web and Android interface layers using standard messaging and APIs. Specialized perceptual and interactive skills are implemented as ROS services for reusable deployment on any robot. This facilitates rapid prototyping of collaborative behaviors that synchronize perception with physical actuation. We experimentally validated core SROS technologies including computer vision, speech processing, and GPT2 autocomplete speech implemented as plug-and-play ROS services. Modularity is demonstrated through the successful integration of an additional ROS package, without changes to hardware or software platforms. The capabilities enabled confirm SROS's effectiveness in developing socially interactive robots through synchronized cross-domain interaction. Through demonstrations showing synchronized multimodal behaviors on an example platform, we illustrate how the SROS architectural approach addresses shortcomings of previous work by lowering barriers for researchers to advance the state-of-the-art in adaptive, collaborative customizable human-robot systems through novel applications integrating perceptual and social abilities.

Introduction

Social robotics is an interdisciplinary field that combines expertise from robotics, AI, computer vision, and interactive design to develop robots capable of engaging in social interactions with humans. A significant challenge in this area includes integrating these diverse technologies into a cohesive platform. To address this issue, a framework called the Socially-interactive Robot Software platform (SROS) was developed, facilitating the rapid prototyping of social robots and emphasizing customizability and modularity.

System Architecture

SROS is structured on a multi-layered architecture to enable advanced social robot features. The Graphical User Interface (GUI) Layer allows users to remotely control and monitor the robot through a web interface. Below that, the Web Layer manages interactions between the GUI and other decision-making layers, also handling database transactions. The ROS Layer, central to the system, oversees robot inputs/outputs, sensor data processing, and actuation while serving as a hub for reusable perception and analysis services. Finally, the Hardware Layer allows SROS to interface with various robotic components like cameras, microphones, and actuators, underscoring the framework's hardware-agnostic nature.

Key Technologies and Demonstration

SROS encompasses several vital technologies crucial for social robot functionality. Its computer vision module employs Media Pipe for real-time landmark detection and DeepFace for facial expression classification. The speech processing pipeline includes features extraction and sentiment analysis utilizing Google Cloud Speech API for Persian speech recognition, reflecting SROS's language support extension. Demonstrations of SROS include coordinating behaviors such as emotion classification, gesture recognition, and gaze estimation across different hardware setups, showcasing the platform's effectiveness and flexibility.

Conclusion

The SROS framework represents an integrated, open-source solution tailored for the development of social robots. Its modular architecture promotes seamless integration of robotics core functions with advanced AI services and user interfaces. Through experimental validation, SROS has proven capable of fostering collaboration, adaptability, and rapid development in social robotics, setting the stage for continued innovation and application in this dynamic field.