Unsupervised Multi-Person 3D Human Pose Estimation From 2D Poses Alone (2309.14865v3)
Abstract: Current unsupervised 2D-3D human pose estimation (HPE) methods do not work in multi-person scenarios due to perspective ambiguity in monocular images. Therefore, we present one of the first studies investigating the feasibility of unsupervised multi-person 2D-3D HPE from just 2D poses alone, focusing on reconstructing human interactions. To address the issue of perspective ambiguity, we expand upon prior work by predicting the cameras' elevation angle relative to the subjects' pelvis. This allows us to rotate the predicted poses to be level with the ground plane, while obtaining an estimate for the vertical offset in 3D between individuals. Our method involves independently lifting each subject's 2D pose to 3D, before combining them in a shared 3D coordinate system. The poses are then rotated and offset by the predicted elevation angle before being scaled. This by itself enables us to retrieve an accurate 3D reconstruction of their poses. We present our results on the CHI3D dataset, introducing its use for unsupervised 2D-3D pose estimation with three new quantitative metrics, and establishing a benchmark for future research.
- Evaluating the impact of wide-angle lens distortion on learning-based depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 3693–3701, 2021.
- Unsupervised 3d pose estimation with geometric self-supervision. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5707–5717, 2019.
- Can 3d pose be learned from 2d projections alone? In Computer Vision – ECCV 2018 Workshops, pages 78–94, Cham, 2019. Springer International Publishing.
- Three-dimensional reconstruction of human interactions. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
- Remips: Physically consistent 3d reconstruction of multiple interacting people under weak supervision. In Advances in Neural Information Processing Systems, pages 19385–19397. Curran Associates, Inc., 2021.
- LInKs - Lifting Independent Keypoints - Exploring Partial Pose Lifting for Occlusion Handling and Improved Accuracy within 2D-3D Human Pose Estimation. In Submitted to WACV 2024 (Submission Version Available Online @ https://github.com/Aswarin/Papers.
- Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7):1325–1339, 2014.
- Coherent reconstruction of multiple humans from a single image. In CVPR, 2020.
- Real-time omnidirectional 3D multi-person human pose estimation with occlusion handling. In Submitted to CVMP 2023 (Submission Version Available Online @ https://github.com/Aswarin/Papers).
- Cliff: Carrying location information in full frames into human pose and shape estimation. In ECCV, 2022.
- Monocular 3d human pose estimation in the wild using improved cnn supervision. In 3D Vision (3DV), 2017 Fifth International Conference on. IEEE, 2017.
- Disambiguating monocular depth estimation with a single transient. In Computer Vision – ECCV 2020, pages 139–155, Cham, 2020. Springer International Publishing.
- Expressive body capture: 3d hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.
- Elepose: Unsupervised 3d human pose estimation by predicting camera elevation and learning normalizing flows on 2d poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6635–6645, 2022.
- Ghum & ghuml: Generative 3d human shape and articulated pose models. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6183–6192, 2020.
- Towards alleviating the modeling ambiguity of unsupervised monocular 3d human pose estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 8651–8660, 2021.