Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Robust 3D Pose Transfer with Adversarial Learning (2404.02242v1)

Published 2 Apr 2024 in cs.CV

Abstract: 3D pose transfer that aims to transfer the desired pose to a target mesh is one of the most challenging 3D generation tasks. Previous attempts rely on well-defined parametric human models or skeletal joints as driving pose sources. However, to obtain those clean pose sources, cumbersome but necessary pre-processing pipelines are inevitable, hindering implementations of the real-time applications. This work is driven by the intuition that the robustness of the model can be enhanced by introducing adversarial samples into the training, leading to a more invulnerable model to the noisy inputs, which even can be further extended to directly handling the real-world data like raw point clouds/scans without intermediate processing. Furthermore, we propose a novel 3D pose Masked Autoencoder (3D-PoseMAE), a customized MAE that effectively learns 3D extrinsic presentations (i.e., pose). 3D-PoseMAE facilitates learning from the aspect of extrinsic attributes by simultaneously generating adversarial samples that perturb the model and learning the arbitrary raw noisy poses via a multi-scale masking strategy. Both qualitative and quantitative studies show that the transferred meshes given by our network result in much better quality. Besides, we demonstrate the strong generalizability of our method on various poses, different domains, and even raw scans. Experimental results also show meaningful insights that the intermediate adversarial samples generated in the training can successfully attack the existing pose transfer models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Skeleton-aware networks for deep motion retargeting. TOG, 39(4):62–1, 2020.
  2. Adversarial robustness for unsupervised domain adaptation. In ICCV, pages 8568–8577, 2021.
  3. Faust: Dataset and evaluation for 3d mesh registration. In CVPR, 2014.
  4. Keep it smpl: Automatic estimation of 3d human pose and shape from a single image. In ECCV, 2016.
  5. Dynamic faust: Registering human bodies in motion. In CVPR, 2017.
  6. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39–57. Ieee, 2017.
  7. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705, 2019.
  8. Aniformer: Data-driven 3d animation with transformer. BMVC, 2021a.
  9. Intrinsic-extrinsic preserved gans for unsupervised 3d pose transfer. In ICCV, pages 8630–8639, 2021b.
  10. Geometry-contrastive transformer for generalized 3d pose transfer. In AAAI, pages 258–266, 2022.
  11. Smg: A micro-gesture dataset towards spontaneous body gestures for emotional stress state analysis. International Journal of Computer Vision, 2023a.
  12. Lart: Neural correspondence learning with latent regularization transformer for 3d motion transfer. NeurIPS, 36, 2024.
  13. Weakly-supervised 3d pose transfer with keypoints. In CVPR, pages 15156–15165, 2023b.
  14. Limp: Learning latent shape representations with metric preservation priors. ECCV, 2020.
  15. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, pages 2206–2216. PMLR, 2020.
  16. Self-robust 3d point recognition via gather-vector guidance. In CVPR, pages 11513–11521, 2020.
  17. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
  18. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
  19. 3d-coded: 3d correspondences by deep deformation. In ECCV, 2018.
  20. Masked autoencoders are scalable vision learners. In CVPR, pages 16000–16009, 2022.
  21. Rda: Robust domain adaptation via fourier adversarial attacking. In ICCV, pages 8988–8999, 2021.
  22. Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, pages 1501–1510, 2017.
  23. Minimal adversarial examples for deep learning on 3d point clouds. In ICCV, pages 7797–7806, 2021.
  24. Adam: A method for stochastic optimization. In ICLR, 2015.
  25. Adaptive adversarial norm space for efficient adversarial training. BMVC, 2023.
  26. Robust structured declarative classifiers for 3d point clouds: Defending adversarial attacks with implicit gradients. In CVPR, pages 15294–15304, 2022.
  27. Skeleton-free pose transfer for stylized 3d characters. In ECCV. Springer, 2022.
  28. Extending adversarial attacks and defenses to deep 3d point cloud classifiers. In ICIP, pages 2279–2283. IEEE, 2019.
  29. Smpl: A skinned multi-person linear model. TOG, 34(6):1–16, 2015.
  30. The power of points for modeling humans in clothing. In ICCV, pages 10974–10984, 2021.
  31. Universal adversarial perturbations. In CVPR, pages 1765–1773, 2017.
  32. Masked autoencoders for point cloud self-supervised learning. arXiv preprint arXiv:2203.06604, 2022.
  33. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, 2019.
  34. Expressive body capture: 3D hands, face, and body from a single image. In CVPR, pages 10975–10985, 2019.
  35. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 652–660, 2017a.
  36. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. NeurIPS, 30, 2017b.
  37. 3d pose transfer with correspondence learning and mesh refinement. NeurIPS, 34, 2021.
  38. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828–841, 2019.
  39. Pointdp: Diffusion-driven purification against adversarial attacks on 3d point cloud recognition. arXiv preprint arXiv:2208.09801, 2022.
  40. Robust adversarial objects against deep learning models. In AAAI, pages 954–962, 2020.
  41. Neural pose transfer by spatially adaptive instance normalization. In CVPR, 2020.
  42. When human pose estimation meets robustness: Adversarial algorithms and benchmarks. In CVPR, pages 11855–11864, 2021a.
  43. Art-point: Improving rotation robustness of point cloud classifiers via adversarial rotation. In CVPR, pages 14371–14380, 2022.
  44. Locally aware piecewise transformation fields for 3d human mesh registration. In CVPR, pages 7639–7648, 2021b.
  45. Metaavatar: Learning animatable clothed human models from few depth images. In NeurIPS, pages 2810–2822, 2021c.
  46. If-defense: 3d adversarial point cloud defense via implicit function based restoration. arXiv preprint arXiv:2010.05272, 2020.
  47. Generating 3d adversarial point clouds. In CVPR, pages 9136–9144, 2019.
  48. Gd-mae: generative decoder for mae pre-training on lidar point clouds. In CVPR, pages 9403–9414, 2023.
  49. Pointr: Diverse point cloud completion with geometry-aware transformers. In ICCV, pages 12498–12507, 2021.
  50. 3d adversarial attacks beyond point cloud. arXiv preprint arXiv:2104.12146, 2021.
  51. Skinned motion retargeting with residual perception of motion semantics & geometry. In CVPR, pages 13864–13872, 2023.
  52. Point-m2ae: Multi-scale masked autoencoders for hierarchical point cloud pre-training. arXiv preprint arXiv:2205.14401, 2022a.
  53. Point-m2ae: multi-scale masked autoencoders for hierarchical point cloud pre-training. NeurIPS, 35:27061–27074, 2022b.
  54. Point transformer. In ICCV, pages 16259–16268, 2021.
  55. Pointcloud saliency maps. In ICCV, pages 1598–1606, 2019.
  56. Unsupervised shape and pose disentanglement for 3d meshes. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16, pages 341–357. Springer, 2020.
  57. Improving robustness of facial landmark detection by defending against adversarial attacks. In ICCV, pages 11751–11760, 2021.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets