Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SuDA: Support-based Domain Adaptation for Sim2Real Motion Capture with Flexible Sensors (2405.16152v1)

Published 25 May 2024 in cs.CV and cs.HC

Abstract: Flexible sensors hold promise for human motion capture (MoCap), offering advantages such as wearability, privacy preservation, and minimal constraints on natural movement. However, existing flexible sensor-based MoCap methods rely on deep learning and necessitate large and diverse labeled datasets for training. These data typically need to be collected in MoCap studios with specialized equipment and substantial manual labor, making them difficult and expensive to obtain at scale. Thanks to the high-linearity of flexible sensors, we address this challenge by proposing a novel Sim2Real Mocap solution based on domain adaptation, eliminating the need for labeled data yet achieving comparable accuracy to supervised learning. Our solution relies on a novel Support-based Domain Adaptation method, namely SuDA, which aligns the supports of the predictive functions rather than the instance-dependent distributions between the source and target domains. Extensive experimental results demonstrate the effectiveness of our method andits superiority over state-of-the-art distribution-based domain adaptation methods in our task.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. i-sim2real: Reinforcement learning of robotic policies in tight human-robot interaction loops. In Conference on Robot Learning. PMLR, 212–224.
  2. A novel flex sensor-based flexible smart garment for monitoring body postures. Journal of Industrial Textiles 49, 2 (2019), 262–274.
  3. Batch fabrication of customizable silicone-textile composite capacitive strain sensors for human motion tracking. Advanced Materials Technologies 2, 9 (2017), 1700136.
  4. Unsupervised pixel-level domain adaptation with generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3722–3731.
  5. DisPad: Flexible On-Body Displacement of Fabric Sensors for Robust Joint-Motion Tracking. arXiv preprint arXiv:2301.06249 (2023).
  6. Full-body human motion reconstruction with sparse joint tracking using flexible sensors. ACM Transactions on Multimedia Computing, Communications and Applications 20, 2 (2023), 1–19.
  7. Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3941–3950.
  8. OpenSim: open-source software to create and analyze dynamic simulations of movement. IEEE transactions on biomedical engineering 54, 11 (2007), 1940–1950.
  9. Adversarial training helps transfer learning via better representations. Advances in Neural Information Processing Systems 34 (2021), 25179–25191.
  10. Carl Doersch and Andrew Zisserman. 2019. Sim2real transfer learning for 3d human pose estimation: motion to the rescue. Advances in Neural Information Processing Systems 32 (2019).
  11. Motsynth: How can synthetic data help pedestrian detection and tracking?. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10849–10859.
  12. Handwriting velcro: Endowing AR glasses with personalized and posture-adaptive text input using flexible touch sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 4 (2023), 1–31.
  13. Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. PMLR, 1180–1189.
  14. Rlsbench: Domain adaptation under relaxed label shift. In International Conference on Machine Learning. PMLR, 10879–10928.
  15. Interactive hand pose estimation using a stretch-sensing soft glove. ACM Transactions on Graphics (ToG) 38, 4 (2019), 1–15.
  16. Optimal kernel choice for large-scale two-sample tests. Advances in neural information processing systems 25 (2012).
  17. Touch-and-Heal: Data-driven Affective Computing in Tactile Interaction with Robotic Dog. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 2 (2023), 1–33.
  18. Sim2Real in robotics and automation: Applications and challenges. IEEE transactions on automation science and engineering 18, 2 (2021), 398–400.
  19. Object detection using sim2real domain randomization for robotic applications. IEEE Transactions on Robotics (2022).
  20. Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–15.
  21. Identifying human body states by using a flexible integrated sensor. npj Flexible Electronics 4, 1 (2020), 1–8.
  22. Dual-sensing, stretchable, fatigue-resistant, adhesive, and conductive hydrogels used as flexible sensors for human motion monitoring. Langmuir 38, 22 (2022), 7013–7023.
  23. Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learning. arXiv preprint arXiv:2303.14870 (2023).
  24. Wireless sensors for continuous, multimodal measurements at the skin interface with lower limb prostheses. Science translational medicine 12, 574 (2020), eabc4327.
  25. Confidence score for source-free unsupervised domain adaptation. In International Conference on Machine Learning. PMLR, 12365–12377.
  26. Selective transfer learning with adversarial training for stock movement prediction. Connection Science 34, 1 (2022), 492–510.
  27. End-to-end human pose and mesh reconstruction with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1954–1963.
  28. Functionalized Fiber-Based Strain Sensors: Pathway to Next-Generation Wearable Electronics. Nano-Micro Letters 14, 1 (2022), 1–39.
  29. Learning transferable features with deep adaptation networks. In International conference on machine learning. PMLR, 97–105.
  30. SMPL: A skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2. 851–866.
  31. Yiyue Luo. 2023. Intelligent Textiles for Physical Human-Environment Interactions. In Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–5.
  32. Learning human–environment interactions using conformal tactile textiles. Nature Electronics 4, 3 (2021), 193–201.
  33. A primer on motion capture with deep learning: principles, pitfalls, and perspectives. Neuron 108, 1 (2020), 44–65.
  34. Mixamo Animation. 2022. https://www.mixamo.com/.
  35. IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–12.
  36. Driving policy transfer via modularity and abstraction. arXiv preprint arXiv:1804.09364 (2018).
  37. Sergey I Nikolenko. 2021. Synthetic data for deep learning. Vol. 174. Springer.
  38. Sinno Jialin Pan and Qiang Yang. 2009. A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22, 10 (2009), 1345–1359.
  39. Sim-to-real transfer learning approach for tracking multi-DOF ankle motions using soft strain sensors. IEEE Robotics and Automation Letters 5, 2 (2020), 3525–3532.
  40. To transfer or not to transfer. In NIPS 2005 workshop on transfer learning, Vol. 898.
  41. X-avatar: Expressive human avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16911–16921.
  42. SynPeDS: A Synthetic Dataset for Pedestrian Detection in Urban Traffic Scenes. In Proceedings of the 6th ACM Computer Science in Cars Symposium. 1–10.
  43. Baochen Sun and Kate Saenko. 2016. Deep coral: Correlation alignment for deep domain adaptation. In European conference on computer vision. Springer, 443–450.
  44. Recovering 3d human mesh from monocular images: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).
  45. Soft smart garments for lower limb joint position analysis. Sensors 17, 10 (2017), 2314.
  46. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014).
  47. Balanced distribution adaptation for transfer learning. In 2017 IEEE international conference on data mining (ICDM). IEEE, 1129–1134.
  48. Characterizing and avoiding negative transfer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11293–11302.
  49. Distribution-Aware Single-Stage Models for Multi-Person 3D Pose Estimation. arXiv:2203.07697 [cs.CV]
  50. Low-Cost, facile, and scalable manufacturing of capacitive sensors for soft systems. Advanced Materials Technologies 2, 9 (2017), 1700072.
  51. QuestSim: Human motion tracking from sparse sensors with simulated avatars. In SIGGRAPH Asia 2022 Conference Papers. 1–8.
  52. Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 4531–4540.
  53. Physical inertial poser (pip): Physics-aware real-time human motion tracking from sparse inertial sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13167–13178.
  54. Transpose: Real-time 3d human translation and pose estimation with six inertial sensors. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–13.
  55. How transferable are features in deep neural networks? Advances in neural information processing systems 27 (2014).
  56. Transfer learning with dynamic adversarial adaptation network. In 2019 IEEE International Conference on Data Mining (ICDM). IEEE, 778–786.
  57. Development and design of flexible sensors used in pressure-monitoring sports pants for human knee joints. IEEE Sensors Journal 21, 22 (2021), 25400–25408.
  58. Recent advances in flexible sensors and their applications. Sensors 22, 12 (2022), 4653.
  59. Sensock: 3D foot reconstruction with flexible sensors. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
  60. Wearable Triboelectric Sensors Enabled Gait Analysis and Waist Motion Capture for IoT-Based Smart Healthcare Applications. Advanced Science 9, 4 (2022), 2103694.
  61. A survey on negative transfer. IEEE/CAA Journal of Automatica Sinica 10, 2 (2022), 305–329.
  62. MoCaPose: Motion Capturing with Textile-integrated Capacitive Sensors in Loose-fitting Smart Garments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 1 (2023), 1–40.
  63. Does computer vision matter for action? Science Robotics 4, 30 (2019), eaaw6661.
  64. Self-adaptive motion tracking against on-body displacement of flexible sensors. (2023).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets