History Repeats Itself: Human Motion Prediction via Motion Attention (2007.11755v1)

Abstract: Human motion prediction aims to forecast future human poses given a past motion. Whether based on recurrent or feed-forward neural networks, existing methods fail to model the observation that human motion tends to repeat itself, even for complex sports actions and cooking activities. Here, we introduce an attention-based feed-forward network that explicitly leverages this observation. In particular, instead of modeling frame-wise attention via pose similarity, we propose to extract motion attention to capture the similarity between the current motion context and the historical motion sub-sequences. Aggregating the relevant past motions and processing the result with a graph convolutional network allows us to effectively exploit motion patterns from the long-term history to predict the future poses. Our experiments on Human3.6M, AMASS and 3DPW evidence the benefits of our approach for both periodical and non-periodical actions. Thanks to our attention model, it yields state-of-the-art results on all three datasets. Our code is available at https://github.com/wei-mao-2019/HisRepItself.

• The paper introduces a novel attention module that leverages motion repetition to enhance human motion prediction.
• It employs a Discrete Cosine Transform for temporal dependency and a Graph Convolutional Network for spatial analysis across joints.
• Experimental results on multiple datasets demonstrate state-of-the-art performance in both periodic and complex non-periodic actions.

An Expert Overview of "History Repeats Itself: Human Motion Prediction via Motion Attention"

The paper "History Repeats Itself: Human Motion Prediction via Motion Attention" addresses the challenge of predicting future human motion by leveraging the recurrent nature of human activities. The authors propose an innovative approach that utilizes an attention-based feed-forward network to model motion prediction more effectively than traditional recurrent or feed-forward neural networks. A critical insight of this work is the observation that human actions tend to recur, not only in periodical movements such as walking but also in complex sequences found in activities like sports and cooking.

Methodology

The methodology centers on exploiting attention mechanisms to improve the predictability of human motion by explicitly modeling motion repetition. Rather than considering individual frame-wise similarities, the paper introduces the concept of "motion attention" to evaluate the likeness between the current motion context and historical motion sub-sequences. This approach eschews conventional pose similarity, opting instead for an aggregation of relevant historical motion sequences. These sequences, processed through a graph convolutional network (GCN), allow for the extraction of motion patterns over long-term histories.

The core idea is implemented via two key components: Motion Attention and the GCN-based Prediction Model. The Motion Attention module calculates attention from short sequences by representing sub-sequences in trajectory space through the Discrete Cosine Transform (DCT). This transformation captures temporal dependencies while the GCN serves to learn spatial dependencies across joint coordinates. The method dynamically adapts attention to the most relevant segments of the historical data, truncated to eliminate high frequencies to prevent jittery predictions. This results in the model effectively capturing the recurring nature of human motion, enabling superior performance across both short and long-term predictions.

Experimental Results

The paper presents robust experimental results on three datasets: Human3.6M, AMASS, and 3DPW. Across these datasets, the proposed method achieves state-of-the-art results in motion prediction. Notably, the approach outperforms existing models such as LTD-50-25, LTD-10-25, and several others in both periodic and complex non-periodic actions. The quantitative evaluations showcase the method's proficiency in both scenarios—especially commendable is the performance on actions featuring clear repetitive historical patterns.

Implications and Future Directions

This work implies significant advancements for applications necessitating accurate motion prediction, including human-robot interaction, surveillance, virtual reality, and animation, where anticipating human movement is crucial. The introduction of motion attention can redirect the development of new predictive models that require capturing complex dependencies in sequential data.

In terms of future work, the authors suggest further exploration into pattern discovery at a more granular level, such as specific limbs or joints, that may promise greater flexibility and the potential to reveal intricate motion-dependent correlations. Additionally, improvements in generalizability could extend this model's applicability across a broader range of untrained datasets, as indicated by its competitive results on new data sequences in 3DPW.

Conclusion

The paper "History Repeats Itself: Human Motion Prediction via Motion Attention" contributes significantly to the field of human motion prediction by introducing a mechanism that captures the recurrence in human actions. The blend of attention mechanics with graph convolutional networks provides a new lens to mediate memory in temporal sequences, setting a high bar for future methodologies in this domain. The model's adaptability and superior performance mark it as a valuable tool for numerous applications, advancing the predictive capabilities of AI systems in interpreting and anticipating human motion.