Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Closed-Loop Multi-perspective Visual Servoing Approach with Reinforcement Learning (2312.15809v1)

Published 25 Dec 2023 in cs.RO and cs.AI

Abstract: Traditional visual servoing methods suffer from serving between scenes from multiple perspectives, which humans can complete with visual signals alone. In this paper, we investigated how multi-perspective visual servoing could be solved under robot-specific constraints, including self-collision, singularity problems. We presented a novel learning-based multi-perspective visual servoing framework, which iteratively estimates robot actions from latent space representations of visual states using reinforcement learning. Furthermore, our approaches were trained and validated in a Gazebo simulation environment with connection to OpenAI/Gym. Through simulation experiments, we showed that our method can successfully learn an optimal control policy given initial images from different perspectives, and it outperformed the Direct Visual Servoing algorithm with mean success rate of 97.0%.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (26)
  1. A. Saxena, H. Pandya, G. Kumar, A. Gaud, and K. M. Krishna, “Exploring convolutional networks for end-to-end visual servoing,” in 2017 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2017, pp. 3817–3823.
  2. Q. Bateux, E. Marchand, J. Leitner, F. Chaumette, and P. Corke, “Training deep neural networks for visual servoing,” in 2018 IEEE international conference on robotics and automation (ICRA).   IEEE, 2018, pp. 3307–3314.
  3. C. Yu, Z. Cai, H. Pham, and Q.-C. Pham, “Siamese convolutional neural network for sub-millimeter-accurate camera pose estimation and visual servoing,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2019, pp. 935–941.
  4. C. Sampedro, A. Rodriguez-Ramos, I. Gil, L. Mejias, and P. Campoy, “Image-based visual servoing controller for multirotor aerial robots using deep reinforcement learning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2018, pp. 979–986.
  5. H. Shi, H. Wu, C. Xu, J. Zhu, M. Hwang, and K.-S. Hwang, “Adaptive image-based visual servoing using reinforcement learning with fuzzy state coding,” IEEE Transactions on Fuzzy Systems, vol. 28, no. 12, pp. 3244–3255, 2020.
  6. P. Singh, V. Singh, S. Dutta, and S. Kumar, “Model & feature agnostic eye-in-hand visual servoing using deep reinforcement learning with prioritized experience replay,” in 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).   IEEE, 2019, pp. 1–8.
  7. T. Lampe and M. Riedmiller, “Acquiring visual servoing reaching and grasping skills using neural reinforcement learning,” in The 2013 international joint conference on neural networks (IJCNN).   IEEE, 2013, pp. 1–8.
  8. H. Shi, X. Li, K.-S. Hwang, W. Pan, and G. Xu, “Decoupled visual servoing with fuzzy q-learning,” IEEE Transactions on Industrial Informatics, vol. 14, no. 1, pp. 241–252, 2016.
  9. Z. Jin, J. Wu, A. Liu, W.-A. Zhang, and L. Yu, “Policy-based deep reinforcement learning for visual servoing control of mobile robots with visibility constraints,” IEEE Transactions on Industrial Electronics, vol. 69, no. 2, pp. 1898–1908, 2021.
  10. M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, O. Pieter Abbeel, and W. Zaremba, “Hindsight experience replay,” Advances in neural information processing systems, vol. 30, 2017.
  11. L. Weiss, A. Sanderson, and C. Neuman, “Dynamic sensor-based control of robots with visual feedback,” IEEE Journal on Robotics and Automation, vol. 3, no. 5, pp. 404–417, 1987.
  12. J. T. Feddema and O. R. Mitchell, “Vision-guided servoing with feature-based trajectory generation (for robots),” IEEE Transactions on Robotics and Automation, vol. 5, no. 5, pp. 691–700, 1989.
  13. W. J. Wilson, C. W. Hulls, and G. S. Bell, “Relative end-effector control using cartesian position based visual servoing,” IEEE Transactions on Robotics and Automation, vol. 12, no. 5, pp. 684–696, 1996.
  14. B. Thuilot, P. Martinet, L. Cordesses, and J. Gallice, “Position based visual servoing: keeping the object in the field of vision,” in Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), vol. 2.   IEEE, 2002, pp. 1624–1629.
  15. F. Chaumette, “Potential problems of stability and convergence in image-based and position-based visual servoing,” in The confluence of vision and control.   Springer, 1998, pp. 66–78.
  16. P. I. Corke and S. A. Hutchinson, “A new partitioned approach to image-based visual servo control,” IEEE Transactions on Robotics and Automation, vol. 17, no. 4, pp. 507–515, 2001.
  17. K. Deguchi, “A direct interpretation of dynamic images with camera and object motions for vision guided robot control,” International Journal of Computer Vision, vol. 37, no. 1, pp. 7–20, 2000.
  18. V. Kallem, M. Dewan, J. P. Swensen, G. Hager, and N. J. Cowan, “Kernel-based visual servoing,” 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1975–1980, 2007.
  19. C. Collewet, E. Marchand, and F. Chaumette, “Visual servoing set free from image processing,” in 2008 IEEE International Conference on Robotics and Automation.   IEEE, 2008, pp. 81–86.
  20. A. Dame and E. Marchand, “Entropy-based visual servoing,” in 2009 IEEE International Conference on Robotics and Automation.   IEEE, 2009, pp. 707–713.
  21. C. Collewet and E. Marchand, “Photometric visual servoing,” IEEE Transactions on Robotics, vol. 27, no. 4, pp. 828–834, 2011.
  22. E. Y. Puang, K. P. Tee, and W. Jing, “Kovis: Keypoint-based visual servoing with zero-shot sim-to-real transfer for robotics manipulation,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   IEEE, 2020, pp. 7527–7533.
  23. S. Felton, E. Fromont, and E. Marchand, “Siame-se (3): regression in se (3) for end-to-end visual servoing,” in 2021 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2021, pp. 14 454–14 460.
  24. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015.
  25. S. Fujimoto, H. van Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80.   PMLR, 10–15 Jul 2018, pp. 1587–1596. [Online]. Available: https://proceedings.mlr.press/v80/fujimoto18a.html
  26. Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 3730–3738.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now