Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Autonomous Field-of-View Adjustment Using Adaptive Kinematic Constrained Control with Robot-Held Microscopic Camera Feedback (2309.10287v2)

Published 19 Sep 2023 in cs.RO

Abstract: Robotic systems for manipulation in millimeter scale often use a camera with high magnification for visual feedback of the target region. However, the limited field-of-view (FoV) of the microscopic camera necessitates camera motion to capture a broader workspace environment. In this work, we propose an autonomous robotic control method to constrain a robot-held camera within a designated FoV. Furthermore, we model the camera extrinsics as part of the kinematic model and use camera measurements coupled with a U-Net based tool tracking to adapt the complete robotic model during task execution. As a proof-of-concept demonstration, the proposed framework was evaluated in a bi-manual setup, where the microscopic camera was controlled to view a tool moving in a pre-defined trajectory. The proposed method allowed the camera to stay 94.1% of the time within the real FoV, compared to 54.4% without the proposed adaptive control.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (27)
  1. H. Koike, K. Iwasawa, R. Ouchi, M. Maezawa, K. Giesbrecht, N. Saiki, A. Ferguson, M. Kimura, W. L. Thompson, J. M. Wells, A. M. Zorn, and T. Takebe, “Modelling human hepato-biliary-pancreatic organogenesis from the foregut–midgut boundary,” Nature, vol. 574, no. 7776, pp. 112–116, Oct. 2019.
  2. M. M. Marinho, J. J. Quiroz-Omaña, and K. Harada, “Design and Validation of a Multi-Arm Robotic Platform for Scientific Exploration,” Oct. 2022.
  3. E. Zhao, M. M. Marinho, and K. Harada, “Autonomous Robotic Drilling System for Mice Cranial Window Creation: An Evaluation with an Egg Model,” Mar. 2023.
  4. G. Guthart and J. Salisbury, “The Intuitive/sup TM/ telesurgery system: overview and application,” in Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065), vol. 1, Apr. 2000, pp. 618–621 vol.1, iSSN: 1050-4729.
  5. S. Gharaaty, T. Shu, A. Joubair, W. F. Xie, and I. A. Bonev, “Online pose correction of an industrial robot using an optical coordinate measure machine system,” International Journal of Advanced Robotic Systems, vol. 15, no. 4, p. 1729881418787915, Jul. 2018.
  6. C. Yu and J. Xi, “Simultaneous and on-line calibration of a robot-based inspecting system,” Robotics and Computer-Integrated Manufacturing, vol. 49, pp. 349–360, Feb. 2018.
  7. M. M. Marinho and B. V. Adorno, “Adaptive Constrained Kinematic Control using Partial or Complete Task-Space Measurements,” IEEE Transactions on Robotics, vol. 38, no. 6, pp. 3498–3513, Dec. 2022.
  8. Y. Koyama, M. M. Marinho, M. Mitsuishi, and K. Harada, “Autonomous Coordinated Control of the Light Guide for Positioning in Vitreoretinal Surgery,” CoRR, vol. abs/2107.11985, 2022.
  9. L. Hsu and P. Aquino, “Adaptive visual tracking with uncertain manipulator dynamics and uncalibrated camera,” in Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304), vol. 2, Dec. 1999, pp. 1248–1253 vol.2, iSSN: 0191-2216.
  10. E. Zergeroglu, D. Dawson, M. de Querioz, and A. Behal, “Vision-based nonlinear tracking controllers with uncertain robot-camera parameters,” IEEE/ASME Transactions on Mechatronics, vol. 6, no. 3, pp. 322–337, Sep. 2001.
  11. C. C. Cheah, C. Liu, and J. J. E. Slotine, “Adaptive Tracking Control for Robots with Unknown Kinematic and Dynamic Properties,” The International Journal of Robotics Research, vol. 25, no. 3, pp. 283–296, Mar. 2006.
  12. C. C. Cheah, S. P. Hou, Y. Zhao, and J.-J. E. Slotine, “Adaptive Vision and Force Tracking Control for Robots With Constraint Uncertainty,” IEEE/ASME Transactions on Mechatronics, vol. 15, no. 3, pp. 389–399, Jun. 2010.
  13. X. Zhong, X. Zhong, and X. Peng, “Robots visual servo control with features constraint employing Kalman-neural-network filtering scheme,” Neurocomputing, vol. 151, pp. 268–277, Mar. 2015.
  14. Z. Wang, Z. Liu, Q. Ma, A. Cheng, Y.-h. Liu, S. Kim, A. Deguet, A. Reiter, P. Kazanzides, and R. H. Taylor, “Vision-Based Calibration of Dual RCM-Based Robot Arms in Human-Robot Collaborative Minimally Invasive Surgery,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 672–679, Apr. 2018.
  15. C. P. Bechlioulis, S. Heshmati-alamdari, G. C. Karras, and K. J. Kyriakopoulos, “Robust Image-Based Visual Servoing With Prescribed Performance Under Field of View Constraints,” IEEE Transactions on Robotics, vol. 35, no. 4, pp. 1063–1070, Aug. 2019.
  16. Z. Miao, H. Zhong, J. Lin, Y. Wang, Y. Chen, and R. Fierro, “Vision-Based Formation Control of Mobile Robots With FOV Constraints and Unknown Feature Depth,” IEEE Transactions on Control Systems Technology, vol. 29, no. 5, pp. 2231–2238, Sep. 2021.
  17. S. M. Ali, L. A. Reisner, B. King, A. Cao, G. Auner, M. Klein, and A. K. Pandya, “Eye gaze tracking for endoscopic camera positioning: an application of a hardware/software interface developed to automate Aesop,” Studies in Health Technology and Informatics, vol. 132, pp. 4–7, 2008.
  18. T. Dardona, S. Eslamian, L. A. Reisner, and A. Pandya, “Remote Presence: Development and Usability Evaluation of a Head-Mounted Display for Camera Control on the da Vinci Surgical System,” Robotics, vol. 8, no. 2, 2019.
  19. B. W. King, L. A. Reisner, A. K. Pandya, A. M. Composto, R. D. Ellis, and M. D. Klein, “Towards an Autonomous Robot for Camera Control During Laparoscopic Surgery,” Journal of Laparoendoscopic & Advanced Surgical Techniques, vol. 23, no. 12, pp. 1027–1030, Dec. 2013.
  20. S. Eslamian, L. A. Reisner, and A. K. Pandya, “Development and evaluation of an autonomous camera control algorithm on the da Vinci Surgical System,” The International Journal of Medical Robotics and Computer Assisted Surgery, vol. 16, no. 2, p. e2036, 2020.
  21. J. J. Ji, S. Krishnan, V. Patel, D. Fer, and K. Goldberg, “Learning 2D Surgical Camera Motion From Demonstrations,” in 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), Aug. 2018, pp. 35–42, iSSN: 2161-8089.
  22. G. Chesi and Y. S. Hung, “Global Path-Planning for Constrained and Optimal Visual Servoing,” IEEE Transactions on Robotics, vol. 23, no. 5, pp. 1050–1060, Oct. 2007.
  23. M. K. Gonzalez, N. A. Theissen, A. Barrios, and A. Archenti, “Online compliance error compensation system for industrial manipulators in contact applications,” Robotics and Computer-Integrated Manufacturing, vol. 76, p. 102305, Aug. 2022.
  24. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” May 2015.
  25. B. V. Adorno, “Robot kinematic modeling and control based on dual quaternion algebra — Part I: Fundamentals,” 2017.
  26. M. M. Marinho, B. V. Adorno, K. Harada, and M. Mitsuishi, “Dynamic Active Constraints for Surgical Robots Using Vector-Field Inequalities,” IEEE Transactions on Robotics, vol. 35, no. 5, pp. 1166–1185, Oct. 2019.
  27. M. M. Marinho, B. V. Adorno, K. Harada, K. Deie, A. Deguet, P. Kazanzides, R. H. Taylor, and M. Mitsuishi, “A unified framework for the teleoperation of surgical robots in constrained workspaces,” in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 2721–2727.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Hung-Ching Lin (2 papers)
  2. Murilo Marques Marinho (7 papers)
  3. Kanako Harada (21 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now