Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation (2403.08716v1)

Published 13 Mar 2024 in cs.RO

Abstract: We introduce DIFFTACTILE, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. In contrast to prior tactile simulators which primarily focus on manipulating rigid bodies and often rely on simplified approximations to model stress and deformations of materials in contact, DIFFTACTILE emphasizes physics-based contact modeling with high fidelity, supporting simulations of diverse contact modes and interactions with objects possessing a wide range of material properties. Our system incorporates several key components, including a Finite Element Method (FEM)-based soft body model for simulating the sensing elastomer, a multi-material simulator for modeling diverse object types (such as elastic, elastoplastic, cables) under manipulation, a penalty-based contact model for handling contact dynamics. The differentiable nature of our system facilitates gradient-based optimization for both 1) refining physical properties in simulation using real-world data, hence narrowing the sim-to-real gap and 2) efficient learning of tactile-assisted grasping and contact-rich manipulation skills. Additionally, we introduce a method to infer the optical response of our tactile sensor to contact using an efficient pixel-based neural module. We anticipate that DIFFTACTILE will serve as a useful platform for studying contact-rich manipulations, leveraging the benefits of dense tactile feedback and differentiable physics. Code and supplementary materials are available at the project website https://difftactile.github.io/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Simulation of vision-based tactile sensors using physics based rendering. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp.  1–7. IEEE, 2021.
  2. Tactile sensors for friction estimation and incipient slip detection—toward dexterous robotic manipulation: A review. IEEE Sensors Journal, 18(22):9049–9064, 2018. doi: 10.1109/JSEN.2018.2868340.
  3. Bidirectional sim-to-real transfer for gelsight tactile sensors with cyclegan. IEEE Robotics and Automation Letters, 7(3):6187–6194, 2022.
  4. Tacchi: A pluggable and low computational cost elastomer deformation simulator for optical tactile sensors. IEEE Robotics and Automation Letters, 8(3):1239–1246, 2023.
  5. Tactile sim-to-real policy transfer via real-to-sim image translation. In Aleksandra Faust, David Hsu, and Gerhard Neumann (eds.), Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research. PMLR, 08–11 Nov 2022. URL https://proceedings.mlr.press/v164/church22a.html.
  6. Pybullet, a python module for physics simulation for games, robotics and machine learning. 2016.
  7. Generation of gelsight tactile images for sim2real learning. IEEE Robotics and Automation Letters, 6(2):4177–4184, 2021.
  8. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp. 1861–1870. PMLR, 2018.
  9. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evolutionary computation, 11(1):1–18, 2003.
  10. Learning to read braille: Bridging the tactile reality gap with diffusion models. arXiv preprint arXiv:2304.01182, 2023.
  11. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM Transactions on Graphics (TOG), 37(4):1–14, 2018.
  12. Difftaichi: Differentiable programming for physical simulation. In International Conference on Learning Representations, 2019.
  13. Plasticinelab: A soft-body manipulation benchmark with differentiable physics. arXiv preprint arXiv:2104.03311, 2021.
  14. Learning by breaking: food fracture anticipation for robotic food manipulation. IEEE Access, 10:99321–99329, 2022.
  15. Ipc-graspsim: Reducing the sim2real gap for parallel-jaw grasping with the incremental potential contact model. In 2022 International Conference on Robotics and Automation (ICRA), pp.  6180–6187. IEEE, 2022.
  16. Pac-nerf: Physics augmented continuum neural radiance fields for geometry-agnostic system identification. arXiv preprint arXiv:2303.05512, 2023.
  17. Tactile gym 2.0: Sim-to-real deep reinforcement learning for comparing low-cost high-resolution robot touch. volume 7 of Proceedings of Machine Learning Research, pp. 10754–10761. IEEE, August 2022. doi: 10.1109/LRA.2022.3195195.
  18. Learning neural constitutive laws from motion observations for generalizable pde dynamics. arXiv preprint arXiv:2304.14369, 2023.
  19. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021.
  20. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021.
  21. Egad! an evolved grasping analysis dataset for diversity and reproducibility in robotic manipulation. IEEE Robotics and Automation Letters, 5(3):4368–4375, 2020.
  22. Position based dynamics. Journal of Visual Communication and Image Representation, 18(2):109–118, 2007.
  23. Sim-to-real for robotic tactile sensing via physics-based simulation and learned latent projections. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp.  6444–6451. IEEE, 2021.
  24. Stable-baselines3: Reliable reinforcement learning implementations. The Journal of Machine Learning Research, 22(1):12348–12355, 2021.
  25. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
  26. Taxim: An example-based simulation model for gelsight tactile sensors. IEEE Robotics and Automation Letters, 7(2):2361–2368, 2022.
  27. Grasp stability prediction with sim-to-real transfer from tactile sensing. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.  7809–7816, 2022. doi: 10.1109/IROS47612.2022.9981863.
  28. Shapemap 3-d: Efficient shape mapping through dense touch and vision. In 2022 International Conference on Robotics and Automation (ICRA), pp.  7073–7080. IEEE, 2022.
  29. SynTouch. https://syntouchinc.com/.
  30. Tacto: A fast, flexible, and open-source simulator for high-resolution vision-based tactile sensors. IEEE Robotics and Automation Letters, 7(2):3930–3937, 2022.
  31. Softzoo: A soft robot co-design benchmark for locomotion in diverse environments. arXiv preprint arXiv:2303.09555, 2023.
  32. Fluidlab: A differentiable environment for benchmarking complex fluid manipulation. In The Eleventh International Conference on Learning Representations, 2022.
  33. An End-to-End Differentiable Framework for Contact-Aware Robot Design. In Proceedings of Robotics: Science and Systems, Virtual, July 2021. doi: 10.15607/RSS.2021.XVII.008.
  34. Efficient tactile simulation with differentiability for robotic manipulation. In Conference on Robot Learning, pp.  1488–1498. PMLR, 2023.
  35. Realtime state estimation with tactile and visual sensing. application to planar manipulation. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp.  7778–7785, 2018. doi: 10.1109/ICRA.2018.8463183.
  36. Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors, 17(12):2762, 2017.
  37. Touching a nerf: Leveraging neural radiance fields for tactile sensory data generation. In Conference on Robot Learning, pp.  1618–1628. PMLR, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Zilin Si (12 papers)
  2. Gu Zhang (33 papers)
  3. Qingwei Ben (9 papers)
  4. Branden Romero (9 papers)
  5. Zhou Xian (17 papers)
  6. Chao Liu (358 papers)
  7. Chuang Gan (195 papers)
Citations (11)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets