Understanding Human Motion and Gestures for Underwater Human-Robot Collaboration (1804.02479v1)

Abstract: In this paper, we present a number of robust methodologies for an underwater robot to visually detect, follow, and interact with a diver for collaborative task execution. We design and develop two autonomous diver-following algorithms, the first of which utilizes both spatial- and frequency-domain features pertaining to human swimming patterns in order to visually track a diver. The second algorithm uses a convolutional neural network-based model for robust tracking-by-detection. In addition, we propose a hand gesture-based human-robot communication framework that is syntactically simpler and computationally more efficient than the existing grammar-based frameworks. In the proposed interaction framework, deep visual detectors are used to provide accurate hand gesture recognition; subsequently, a finite-state machine performs robust and efficient gesture-to-instruction mapping. The distinguishing feature of this framework is that it can be easily adopted by divers for communicating with underwater robots without using artificial markers or requiring memorization of complex language rules. Furthermore, we validate the performance and effectiveness of the proposed methodologies through extensive field experiments in closed- and open-water environments. Finally, we perform a user interaction study to demonstrate the usability benefits of our proposed interaction framework compared to existing methods.

• The paper presents two diver-following algorithms, one using MDPM with 84.2%-91.7% accuracy and a CNN model achieving a 97.12% detection rate.
• It proposes an efficient hand gesture-based communication framework that replaces complex syntax with intuitive commands mapped via a finite-state machine.
• The findings enhance underwater human-robot collaboration, enabling real-time interaction and paving the way for advanced mission programming.

An Assessment of Methods for Enhancing Underwater Human-Robot Interaction

The paper "Understanding Human Motion and Gestures for Underwater Human-Robot Collaboration" addresses the complex challenge of underwater human-robot interaction. This research explores robust methodologies enabling underwater robots to detect, follow, and interact with human divers. Two diver-following algorithms are introduced: one that exploits spatial- and frequency-domain features while the other leverages convolutional neural networks (CNNs) to achieve tracking-by-detection. Additionally, the researchers propose a hand gesture-based communication framework, providing simpler syntax while being computationally efficient compared to grammar-based frameworks.

Diver-Following Algorithms

The diver-following problem is tackled using two distinct approaches:

1. Mixed Domain Periodic Motion (MDPM) Tracker: By combining spatial and frequency domain analysis, this algorithm detects human swimming patterns with high efficiency. The MDPM uses Hidden Markov Models for initial motion direction predictions based on windowed intensity values, then refines these predictions via Fourier transforms to identify periodic motion signature resembling human swimming. Field evaluations report positive detection accuracy between 84.2% and 91.7%, suggesting its reliability in various underwater conditions.
2. CNN-Based Diver Detection Model: This method addresses MDPM's limitations regarding detection robustness. A trained CNN offers a scalable solution invariant to diver swimming styles, color of attire, and other appearance factors, achieving an average intersection-over-union (IOU) score of 0.674 with a positive detection rate of 97.12%. Despite slower operation than MDPM, the CNN model's robustness under diverse conditions ensures applicability in real-world implementations.

Hand Gesture-Based Human-Robot Interaction

The communication framework proposed expands underwater robot programmability via intuitive hand gestures without requiring divers to memorize complex sets of language rules or carry fiducial markers. Simple yet distinct gestures map to specific task-switching and parameter reconfiguration instructions through deterministic finite-state machine (FSM) models. This design enhances user experience by alleviating traditional limitations associated with underwater communications where electromagnetic interference often prevails.

Methodologies for Gesture Recognition:

The research employs two cutting-edge deep visual detectors—Faster RCNN with Inception V2 and SSD with MobileNet V2—in addition to an internally developed CNN model. These recognizers are used in real-time for robust hand gesture detection, ensuring correct mapping of gesture-tokens to instruction-tokens despite environmental challenges such as surface reflections and suspended particles.

Implications and Future Directions

The practical implications of this research are significant, particularly for enhancing operational efficiencies during underwater missions by reducing dependency on surface interruptions and complex programming syntaxes. This paper suggests promising pathways toward more effective collaboration between humans and robots within the challenging constraints posed by underwater environments.

For future developments, the authors propose investigating real-time diver pose detection, which could allow robots to anticipate divers' movements and actions. Furthermore, integrating control flow operations could enable more complex mission programming within the proposed human-robot interaction framework, expanding vocabulary and instruction capabilities.

In conclusion, the methodologies presented optimize underwater human-robot collaboration by harnessing visual sensing technologies, which promises valuable advances for underwater exploration and autonomous task execution despite prevalent environmental constraints.