Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

JIGGLE: An Active Sensing Framework for Boundary Parameters Estimation in Deformable Surgical Environments (2405.09743v1)

Published 16 May 2024 in cs.RO

Abstract: Surgical automation can improve the accessibility and consistency of life saving procedures. Most surgeries require separating layers of tissue to access the surgical site, and suturing to reattach incisions. These tasks involve deformable manipulation to safely identify and alter tissue attachment (boundary) topology. Due to poor visual acuity and frequent occlusions, surgeons tend to carefully manipulate the tissue in ways that enable inference of the tissue's attachment points without causing unsafe tearing. In a similar fashion, we propose JIGGLE, a framework for estimation and interactive sensing of unknown boundary parameters in deformable surgical environments. This framework has two key components: (1) a probabilistic estimation to identify the current attachment points, achieved by integrating a differentiable soft-body simulator with an extended Kalman filter (EKF), and (2) an optimization-based active control pipeline that generates actions to maximize information gain of the tissue attachments, while simultaneously minimizing safety costs. The robustness of our estimation approach is demonstrated through experiments with real animal tissue, where we infer sutured attachment points using stereo endoscope observations. We also demonstrate the capabilities of our method in handling complex topological changes such as cutting and suturing.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Active mapping via gradient ascent optimization of shannon mutual information over continuous se(3) trajectories. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 12994–13001, 2022. doi: 10.1109/IROS47612.2022.9981875.
  2. Identification and active exploration of deformable object boundary constraints through robotic manipulation. The International journal of robotics research, 33(11):1446–1461, 2014.
  3. Virtual elastic objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15827–15837, 2022.
  4. Markerless suture needle 6d pose tracking with robust uncertainty estimation for autonomous minimally invasive robotic surgery. CoRR, abs/2109.12722, 2021. URL https://arxiv.org/abs/2109.12722.
  5. Diffpd: Differentiable projective dynamics. ACM Trans. Graph., 41(2), nov 2021. ISSN 0730-0301. doi: 10.1145/3490168. URL https://doi.org/10.1145/3490168.
  6. Sim-to-real for soft robots using differentiable fem: Recipes for meshing, damping, and actuation. IEEE Robotics and Automation Letters, 7(2):5015–5022, 2022. doi: 10.1109/LRA.2022.3154050.
  7. Tumor localization using automated palpation with gaussian process adaptive sampling. In 2016 IEEE International Conference on Automation Science and Engineering (CASE), pages 194–200. IEEE, 2016.
  8. Autonomous system for tumor resection (astr) - dual-arm robotic midline partial glossectomy. IEEE Robotics and Automation Letters, 9(2):1166–1173, 2024. doi: 10.1109/LRA.2023.3341773.
  9. Task-oriented active sensing via action entropy minimization. IEEE Access, 7:135413–135426, 2019.
  10. Disect: A differentiable simulation engine for autonomous robotic cutting. arXiv preprint arXiv:2105.12244, 2021.
  11. Probabilistic inference of simulation parameters via parallel differentiable simulation. In 2022 International Conference on Robotics and Automation (ICRA), pages 3638–3645. IEEE, 2022.
  12. Difftaichi: Differentiable programming for physical simulation. arXiv preprint arXiv:1910.00935, 2019a.
  13. 3-d deformable object manipulation using deep neural networks. IEEE Robotics and Automation Letters, 4(4):4255–4261, 2019b.
  14. Mutual information-based exploration on continuous occupancy maps. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6086–6092. IEEE, 2015.
  15. An open-source research kit for the da vinci® surgical system. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 6434–6439, 2014a. doi: 10.1109/ICRA.2014.6907809.
  16. An open-source research kit for the da vinci® surgical system. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 6434–6439, 2014b. doi: 10.1109/ICRA.2014.6907809.
  17. Autonomous multilateral debridement with the raven surgical robot. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 1432–1439, 2014. doi: 10.1109/ICRA.2014.6907040.
  18. Segment anything. arXiv:2304.02643, 2023.
  19. Multi-robot active slam with relative entropy optimization. In 2013 American Control Conference, pages 2757–2764, 2013. doi: 10.1109/ACC.2013.6580252.
  20. Super: A surgical perception framework for endoscopic tissue manipulation with surgical robotics. IEEE Robotics and Automation Letters, 5(2):2294–2301, April 2020. ISSN 2377-3774. doi: 10.1109/lra.2020.2970659. URL http://dx.doi.org/10.1109/LRA.2020.2970659.
  21. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM Trans. Graph., 42(1), oct 2022. ISSN 0730-0301. doi: 10.1145/3527660. URL https://doi.org/10.1145/3527660.
  22. Real-to-sim deformable object manipulation: Optimizing physics models with residual mappings for robotic surgery. arXiv preprint arXiv:2309.11656, 2023.
  23. Semantic-super: A semantic-aware surgical perception framework for endoscopic tissue identification, reconstruction, and tracking. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 4739–4746, 2023. doi: 10.1109/ICRA48891.2023.10160746.
  24. Raft-stereo: Multilevel recurrent field transforms for stereo matching. In International Conference on 3D Vision (3DV), 2021.
  25. Real-to-sim registration of deformable soft tissue with position-based dynamics for surgical robot autonomy. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 12328–12334, 2021. doi: 10.1109/ICRA48506.2021.9561177.
  26. Robotic manipulation of deformable rope-like objects using differentiable compliant position-based dynamics. IEEE Robotics and Automation Letters, 8(7):3964–3971, 2023. doi: 10.1109/LRA.2023.3264766.
  27. E-DSSR: efficient dynamic surgical scene reconstruction with transformer-based stereoscopic depth perception. CoRR, abs/2107.00229, 2021. URL https://arxiv.org/abs/2107.00229.
  28. Xpbd: position-based simulation of compliant constrained dynamics. In Proceedings of the 9th International Conference on Motion in Games, pages 49–54, 2016.
  29. A comparison of decision making criteria and optimization methods for active robotic sensing. In Numerical Methods and Applications: 5th International Conference, NMA 2002 Borovets, Bulgaria, August 20–24, 2002 Revised Papers 5, pages 316–324. Springer, 2003.
  30. Trajectory optimization for continuous ergodic exploration on the motion group se(2). In 52nd IEEE Conference on Decision and Control, pages 4517–4522, 2013. doi: 10.1109/CDC.2013.6760585.
  31. Position based dynamics. Journal of Visual Communication and Image Representation, 18(2):109–118, 2007.
  32. A surgical robotic framework for safe and autonomous data-driven learning and manipulation of an unknown deformable tissue with an integrated critical space. Journal of Medical Robotics Research, 8(01n02):2340001, 2023.
  33. Model-based identification of anatomical boundary conditions in living tissues. In International Conference on information processing in computer-assisted interventions, pages 196–205. Springer, 2014.
  34. Autonomous robotic suction to clear the surgical field for hemostasis using image-based blood flow detection. IEEE Robotics and Automation Letters, 6(2):1383–1390, 2021. doi: 10.1109/LRA.2021.3056057.
  35. Particle filter based information-theoretic active sensing. Robotics and Autonomous Systems, 58(5):574–584, 2010. ISSN 0921-8890. doi: https://doi.org/10.1016/j.robot.2010.01.001. URL https://www.sciencedirect.com/science/article/pii/S0921889010000023.
  36. Autonomous laparoscopic robotic suturing with a novel actuated suturing tool and 3d endoscope. In 2019 International Conference on Robotics and Automation (ICRA), pages 1541–1547, 2019. doi: 10.1109/ICRA.2019.8794306.
  37. Trajectory-optimized sensing for active search of tissue abnormalities in robotic surgery. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 5356–5363, 2018. doi: 10.1109/ICRA.2018.8460936.
  38. Anytime planning for decentralized multirobot active information gathering. IEEE Robotics and Automation Letters, 3(2):1025–1032, 2018. doi: 10.1109/LRA.2018.2794608.
  39. Active exploration for robotic manipulation. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9355–9362, 2022. doi: 10.1109/IROS47612.2022.9982061.
  40. Supervised autonomous robotic soft tissue surgery. Science Translational Medicine, 8(337):337ra64–337ra64, 2016. doi: 10.1126/scitranslmed.aad9398. URL https://www.science.org/doi/abs/10.1126/scitranslmed.aad9398.
  41. Diffxpbd: Differentiable position-based simulation of compliant constraint dynamics. Proc. ACM Comput. Graph. Interact. Tech., 6(3), aug 2023. doi: 10.1145/3606923. URL https://doi.org/10.1145/3606923.
  42. Online algorithms for pomdps with continuous state, action, and observation spaces. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 28, pages 259–263, 2018.
  43. Diffcloud: Real-to-sim from point clouds with differentiable simulation and rendering of deformable objects. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10828–10835, 2022. doi: 10.1109/IROS47612.2022.9981101.
  44. Data-driven intra-operative estimation of anatomical attachments for autonomous tissue dissection. IEEE Robotics and Automation Letters, 6(2):1856–1863, 2021a. doi: 10.1109/LRA.2021.3060655.
  45. Intra-operative update of boundary conditions for patient-specific surgical simulation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, pages 373–382. Springer, 2021b.
  46. Learning visual shape control of novel 3d deformable objects from partial-view point clouds. In 2022 International Conference on Robotics and Automation (ICRA), pages 8274–8281. IEEE, 2022.
  47. Multilateral surgical pattern cutting in 2d orthotropic gauze with deep reinforcement learning policies for tensioning. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 2371–2378, 2017. doi: 10.1109/ICRA.2017.7989275.
  48. Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery. In International conference on medical image computing and computer-assisted intervention, pages 431–441. Springer, 2022.
  49. Identification of young’s modulus and equivalent spring constraint boundary conditions of the soft tissue with locally observed displacements for endoscopic liver surgery. Computer Methods in Biomechanics and Biomedical Engineering, 25(4):439–454, 2022.
  50. Efficient sampling for rapid estimation of 3-d stiffness distribution via active tactile exploration. IEEE/ASME Transactions on Mechatronics, 25(4):1729–1738, 2020. doi: 10.1109/TMECH.2020.2992573.
  51. Achieving autonomous cloth manipulation with optimal control via differentiable physics-aware regularization and safety constraints. arXiv preprint arXiv:2309.11655, 2023.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now