Simultaneous Learning of Contact and Continuous Dynamics (2310.12054v1)
Abstract: Robotic manipulation can greatly benefit from the data efficiency, robustness, and predictability of model-based methods if robots can quickly generate models of novel objects they encounter. This is especially difficult when effects like complex joint friction lack clear first-principles models and are usually ignored by physics simulators. Further, numerically-stiff contact dynamics can make common model-building approaches struggle. We propose a method to simultaneously learn contact and continuous dynamics of a novel, possibly multi-link object by observing its motion through contact-rich trajectories. We formulate a system identification process with a loss that infers unmeasured contact forces, penalizing their violation of physical constraints and laws of motion given current model parameters. Our loss is unlike prediction-based losses used in differentiable simulation. Using a new dataset of real articulated object trajectories and an existing cube toss dataset, our method outperforms differentiable simulation and end-to-end alternatives with more data efficiency. See our project page for code, datasets, and media: https://sites.google.com/view/continuous-contact-nets/home
- Consensus complementarity control for multi-contact mpc. arXiv preprint arXiv:2304.11259, Apr. 2023.
- A. Aydinoglu and M. Posa. Real-time multi-contact model predictive control via admm. In 2022 International Conference on Robotics and Automation (ICRA), pages 3414–3421. IEEE, 2022.
- L. Ljung. Perspectives on system identification. Annual Reviews in Control, 34(1):1–12, 2010.
- Fundamental challenges in deep learning for stiff contact dynamics. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5181–5188. IEEE, 2021.
- End-to-end differentiable physics for learning and control. Advances in neural information processing systems, 31:7178–7189, 2018.
- Parameter and contact force estimation of planar rigid-bodies undergoing frictional contact. The International Journal of Robotics Research, 36(13-14):1437–1454, 2017.
- ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations. In The Conference on Robot Learning (CoRL), 2020. URL https://proceedings.mlr.press/v155/pfrommer21a.html.
- C. Rucker and P. M. Wensing. Smooth parameterization of rigid-body inertia. IEEE Robotics and Automation Letters, 7(2):2771–2778, 2022.
- Generalization bounded implicit learning of nearly discontinuous functions. In Learning for Dynamics and Control Conference, pages 1112–1124. PMLR, 2022.
- Learning contact dynamics using physically structured neural networks. In International Conference on Artificial Intelligence and Statistics, pages 2152–2160. PMLR, 2021.
- Learning rigid dynamics with face interaction graph networks. arXiv preprint arXiv:2212.03574, 2022.
- Implicit behavioral cloning. In Conference on Robot Learning, pages 158–168. PMLR, 2022.
- Learning data-efficient rigid-body contact models: Case study of planar impact. In Conference on Robot Learning, pages 388–397. PMLR, 2017.
- Differentiating through a cone program. arXiv preprint arXiv:1904.09043, 2019.
- B. Amos and J. Z. Kolter. Optnet: Differentiable optimization as a layer in neural networks. In International Conference on Machine Learning, pages 136–145. PMLR, 2017.
- Single-level differentiable contact simulation. IEEE Robotics and Automation Letters, 2023.
- Dojo: A differentiable simulator for robotics. arXiv preprint arXiv:2203.00806, 2022.
- Rethinking optimization with differentiable simulation from a global perspective. In Conference on Robot Learning, pages 276–286. PMLR, 2023.
- R. Tedrake. Underactuated Robotics. 2023. URL https://underactuated.csail.mit.edu.
- Encoding physical constraints in differentiable newton-euler algorithm. In Learning for Dynamics and Control, pages 804–813. PMLR, 2020.
- Estimation of inertial parameters of manipulator loads and links. The International Journal of Robotics Research, 5(3):101–119, 1986.
- Oscar: Data-driven operational space control for adaptive and robust robot manipulation. In 2022 International Conference on Robotics and Automation (ICRA), pages 10519–10526. IEEE, 2022.
- Neuralsim: Augmenting differentiable simulators with neural networks. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 9474–9481. IEEE, 2021.
- Using inaccurate models in reinforcement learning. In Proceedings of the 23rd international conference on Machine learning, pages 1–8, 2006.
- Augmenting physical simulators with stochastic neural networks: Case study of planar pushing and bouncing. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3066–3073. IEEE, 2018.
- Tossingbot: Learning to throw arbitrary objects with residual physics. IEEE Transactions on Robotics, 36(4):1307–1319, 2020.
- M. Anitescu and F. A. Potra. Formulating dynamic multi-rigid-body contact problems with friction as solvable linear complementarity problems. Nonlinear Dynamics, 14(3):231–247, 1997.
- An implicit time-stepping scheme for rigid body dynamics with inelastic collisions and coulomb friction. International Journal for Numerical Methods in Engineering, 39(15):2673–2691, 1996.
- An unconstrained convex formulation of compliant contact. IEEE Transactions on Robotics, 2022.
- Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE, 2012.
- Validating robotics simulators on real-world impacts. IEEE Robotics and Automation Letters, 7(3):6471–6478, 2022.
- Deep dynamics models for learning dexterous manipulation. In Conference on Robot Learning, pages 1101–1112. PMLR, 2020.
- A direct method for trajectory optimization of rigid bodies through contact. The International Journal of Robotics Research, 33(1):69–81, 2014.
- B. Pfrommer and K. Daniilidis. Tagslam: Robust slam with fiducial markers. arXiv preprint arXiv:1910.00679, 2019.
- Drake: Model-based design and verification for robotics, 2019. URL https://drake.mit.edu.
- M. Anitescu. Optimization-based simulation of nonsmooth rigid multibody dynamics. Mathematical Programming, 105:113–143, 2006.
- Graph network simulators can learn discontinuous, rigid contact dynamics. In Conference on Robot Learning, pages 1157–1167. PMLR, 2023.
- A probabilistic framework for learning kinematic models of articulated objects. Journal of Artificial Intelligence Research, 41:477–526, 2011.
- Sim2real2: Actively building explicit physics model for precise articulated object manipulation. arXiv preprint arXiv:2302.10693, 2023.
- Carto: Category and joint agnostic reconstruction of articulated objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21201–21210, 2023.
- B. Wen and K. Bekris. Bundletrack: 6d pose tracking for novel objects without instance or category-level 3d models. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8067–8074. IEEE, 2021.
- Texpose: Neural texture learning for self-supervised 6d object pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4841–4852, 2023.
- Gen6d: Generalizable model-free 6-dof object pose estimation from rgb images. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXII, pages 298–315. Springer, 2022.