CLOAF: CoLlisiOn-Aware Human Flow

Abstract: Even the best current algorithms for estimating body 3D shape and pose yield results that include body self-intersections. In this paper, we present CLOAF, which exploits the diffeomorphic nature of Ordinary Differential Equations to eliminate such self-intersections while still imposing body shape constraints. We show that, unlike earlier approaches to addressing this issue, ours completely eliminates the self-intersections without compromising the accuracy of the reconstructions. Being differentiable, CLOAF can be used to fine-tune pose and shape estimation baselines to improve their overall performance and eliminate self-intersections in their predictions. Furthermore, we demonstrate how our CLOAF strategy can be applied to practically any motion field induced by the user. CLOAF also makes it possible to edit motion to interact with the environment without worrying about potential collision or loss of body-shape prior.

• The paper introduces CLOAF, a novel ODE-based method that prevents self-intersections in 3D human pose and shape estimation.
• It integrates SMPL body models into an ODE formulation, ensuring continuous and geometrically constrained deformations.
• CLOAF enhances pose estimation via differentiable end-to-end training, promising realistic motion for applications in animation and robotics.

CoLlisiOn-Aware Human Flow: A New Approach to Eliminating Self-Intersections in 3D Human Pose and Shape Estimation

Introduction

The estimation of 3D human pose and shape from single images has made significant strides, with state-of-the-art methods such as transformer-based architectures achieving impressive accuracies. However, these models, even those pre-trained on extensive image datasets and fine-tuned on 3D body shape datasets, sometimes produce estimates with self-intersections of body parts. These unrealistic poses pose challenges in applications like robotics and animation, where maintaining realistic human motions is crucial. In this context, we introduce a novel approach, CoLlisiOn-Aware Flow (CLOAF), which leverages the properties of Ordinary Differential Equations (ODEs) to eliminate self-intersections in a differentiable and geometrically constrained manner.

Current Approaches to Handling Self-Intersections

Existing methods to address the issue of self-intersections in body shape and pose estimation primarily adopt iterative optimization techniques. These include penalizing self-intersections explicitly through loss functions or modifying the training data to reduce the likelihood of self-intersections. However, these approaches either lack differentiability, precluding their use in training deep networks, or do not offer guarantees against self-intersections at inference time.

The CLOAF Method

CLOAF operates by formulating the body shape deformation process as the solution to an ODE, inherently preventing self-intersections due to the diffeomorphic nature of ODE solutions. Given an initial body representation devoid of self-intersections, CLOAF can interpolate towards a target pose, ensuring that all intermediate poses adhere to realistic body shape constraints without self-intersections.

At the core of CLOAF is the integration of a body model, such as SMPL, into the ODE formulation. This integration allows us to compute trajectories in the parameter space of the body model, ensuring the imposition of the body shape prior throughout the deformation process. Through differentiable operations, CLOAF can be incorporated into the training pipelines of pose estimation networks to enhance their performance and reduce self-intersections.

Practical Implications and Future Directions

Our approach is not only capable of refining single-frame pose estimations to eliminate self-intersections but also introduces a methodology that can be integrated into the pose estimation networks' training process. This ability to refine and improve baseline methods through end-to-end training represents a significant advance in the pursuit of more accurate and realistic human pose and shape estimation techniques.

Moreover, the flexibility of CLOAF allows it to be applied to various motion fields induced by the user, paving the way for interactive applications where body movements can be edited in a collision-aware manner. This includes scenarios where human figures interact with environments or objects in a physically plausible way.

Conclusion

In summary, CLOAF represents an innovative step forward in addressing the persistent challenge of self-intersections in 3D human pose and shape estimation. By leveraging the properties of ODEs within a differentiable framework that respects geometric body shape constraints, CLOAF offers a robust solution that enhances the realism and accuracy of pose estimation models. Moving forward, the potential applications of CLOAF in the realms of animation, virtual reality, and robotics are vast and promising, heralding a new era of interaction between digital human representations and their environments.