Papers
Topics
Authors
Recent
Search
2000 character limit reached

Optical Tactile Sensing for Aerial Multi-Contact Interaction: Design, Integration, and Evaluation

Published 30 Jan 2024 in cs.RO | (2401.17149v1)

Abstract: Distributed tactile sensing for multi-force detection is crucial for various aerial robot interaction tasks. However, current contact sensing solutions on drones only exploit single end-effector sensors and cannot provide distributed multi-contact sensing. Designed to be easily mounted at the bottom of a drone, we propose an optical tactile sensor that features a large and curved soft sensing surface, a hollow structure and a new illumination system. Even when spaced only 2 cm apart, multiple contacts can be detected simultaneously using our software pipeline, which provides real-world quantities of 3D contact locations (mm) and 3D force vectors (N), with an accuracy of 1.5 mm and 0.17 N respectively. We demonstrate the sensor's applicability and reliability onboard and in real-time with two demos related to i) the estimation of the compliance of different perches and subsequent re-alignment and landing on the stiffer one, and ii) the mapping of sparse obstacles. The implementation of our distributed tactile sensor represents a significant step towards attaining the full potential of drones as versatile robots capable of interacting with and navigating within complex environments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. D. Floreano and R. Wood, “Science, technology and the future of small autonomous drones,” Nature, vol. 521, pp. 460–6, 05 2015.
  2. J. Delmerico, S. Mintchev, A. Giusti, B. Gromov, K. Melo, T. Horvat, C. Cadena, M. Hutter, A. Ijspeert, D. Floreano, L. M. Gambardella, R. Siegwart, and D. Scaramuzza, “The current state and future outlook of rescue robotics,” Journal of Field Robotics, vol. 36, no. 7, pp. 1171–1191, 2019.
  3. S. Asseng and F. Asche, “Future farms without farmers,” Science Robotics, vol. 4, no. 27, p. eaaw1875, 2019.
  4. T. Petso and R. S. Jamisola, “Wildlife conservation using drones and artificial intelligence in africa,” Science Robotics, vol. 8, no. 85, p. eadm7008, 2023.
  5. A. Loquercio, E. Kaufmann, R. Ranftl, M. Müller, V. Koltun, and D. Scaramuzza, “Learning high-speed flight in the wild,” Science Robotics, vol. 6, no. 59, p. eabg5810, 2021.
  6. X. Zhou, X. Wen, Z. Wang, Y. Gao, H. Li, Q. Wang, T. Yang, H. Lu, Y. Cao, C. Xu, and F. Gao, “Swarm of micro flying robots in the wild,” Science Robotics, vol. 7, no. 66, p. eabm5954, 2022.
  7. C. Geckeler, E. Aucone, Y. Schnider, A. Simeon, J.-P. v. Bassewitz, Y. Zhu, and S. Mintchev, “Learning occluded branch depth maps in forest environments using rgb-d images,” IEEE Robotics and Automation Letters, pp. 1–8, 2024.
  8. A. Briod, P. Kornatowski, J.-C. Zufferey, and D. Floreano, “A collision-resilient flying robot,” Journal of Field Robotics, vol. 31, no. 4, pp. 496–509, 2014.
  9. N. Khedekar, F. Mascarich, C. Papachristos, T. Dang, and K. Alexis, “Contact–based navigation path planning for aerial robots,” in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 4161–4167.
  10. K. Bodie, M. Brunner, M. Pantic, S. Walser, P. Pfändler, U. Angst, R. Siegwart, and J. Nieto, “Active interaction force control for contact-based inspection with a fully actuated aerial vehicle,” IEEE Transactions on Robotics, vol. 37, no. 3, pp. 709–722, 2021.
  11. H. W. Wopereis, W. L. W. van de Ridder, T. J. W. Lankhorst, L. Klooster, E. M. Bukai, D. Wuthier, G. Nikolakopoulos, S. Stramigioli, J. B. C. Engelen, and M. Fumagalli, “Multimodal aerial locomotion: An approach to active tool handling,” IEEE Robotics &\&& Automation Magazine, vol. 25, no. 4, pp. 57–65, 2018.
  12. S. Hamaza, I. Georgilas, M. Fernandez, P. Sanchez, T. Richardson, G. Heredia, and A. Ollero, “Sensor installation and retrieval operations using an unmanned aerial manipulator,” IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2793–2800, 2019.
  13. C. Geckeler and S. Mintchev, “Bistable helical origami gripper for sensor placement on branches,” Advanced Intelligent Systems, vol. 4, no. 10, p. 2200087, 2022.
  14. E. Aucone, S. Kirchgeorg, A. Valentini, L. Pellissier, K. Deiner, and S. Mintchev, “Drone-assisted collection of environmental dna from tree branches for biodiversity monitoring,” Science Robotics, vol. 8, no. 74, p. eadd5762, 2023.
  15. R. S. Dahiya, G. Metta, M. Valle, and G. Sandini, “Tactile sensing—from humans to humanoids,” IEEE Transactions on Robotics, vol. 26, no. 1, pp. 1–20, 2010.
  16. H. Yousef, M. Boukallel, and K. Althoefer, “Tactile sensing for dexterous in-hand manipulation in robotics—a review,” Sensors and Actuators A: Physical, vol. 167, no. 2, pp. 171–187, 2011, solid-State Sensors, Actuators and Microsystems Workshop.
  17. R. S. Dahiya, P. Mittendorfer, M. Valle, G. Cheng, and V. J. Lumelsky, “Directions toward effective utilization of tactile skin: A review,” IEEE Sensors Journal, vol. 13, no. 11, pp. 4121–4138, 2013.
  18. C. Bartolozzi, L. Natale, F. Nori, and G. Metta, “Robots with a sense of touch,” Nature Materials, vol. 15, pp. 921–925, 2016.
  19. C. Fox, M. Evans, M. Pearson, and T. Prescott, “Tactile slam with a biomimetic whiskered robot,” in 2012 IEEE International Conference on Robotics and Automation, 2012, pp. 4925–4930.
  20. H. Kolvenbach, C. Bärtschi, L. Wellhausen, R. Grandia, and M. Hutter, “Haptic inspection of planetary soils with legged robots,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1626–1632, 2019.
  21. A. Vangen, T. Barnwal, J. A. Olsen, and K. Alexis, “Terrain recognition and contact force estimation through a sensorized paw for legged robots,” 2023.
  22. A. Ollero, M. Tognon, A. Suarez, D. Lee, and A. Franchi, “Past, present, and future of aerial robotic manipulators,” IEEE Transactions on Robotics, vol. 38, no. 1, pp. 626–645, 2022.
  23. M. A. Trujillo, J. R. Martínez-de Dios, C. Martín, A. Viguria, and A. Ollero, “Novel aerial manipulator for accurate and robust industrial ndt contact inspection: A new tool for the oil and gas inspection industry,” Sensors, vol. 19, no. 6, 2019.
  24. M. Fumagalli, R. Naldi, A. Macchelli, F. Forte, A. Q. Keemink, S. Stramigioli, R. Carloni, and L. Marconi, “Developing an aerial manipulator prototype: Physical interaction with the environment,” IEEE Robotics & Automation Magazine, vol. 21, no. 3, pp. 41–50, Sep. 2014.
  25. T. Bartelds, A. Capra, S. Hamaza, S. Stramigioli, and M. Fumagalli, “Compliant aerial manipulators: Toward a new generation of aerial robotic workers,” IEEE Robotics and Automation Letters, vol. 1, no. 1, pp. 477–483, Jan. 2016.
  26. G. Nava, Q. Sablé, M. Tognon, D. Pucci, and A. Franchi, “Direct force feedback control and online multi-task optimization for aerial manipulators,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 331–338, Apr. 2020.
  27. D. Lee, H. Seo, I. Jang, S. J. Lee, and H. J. Kim, “Aerial manipulator pushing a movable structure using a dob-based robust controller,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 723–730, 2021.
  28. F. Benzi, M. Brunner, M. Tognon, C. Secchi, and R. Siegwart, “Adaptive tank-based control for aerial physical interaction with uncertain dynamic environments using energy-task estimation,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 9129–9136, 2022.
  29. E. Aucone, C. Geckeler, D. Morra, L. Pallottino, and S. Mintchev, “Synergistic morphology and feedback control for traversal of unknown compliant obstacles with aerial robots,” Preprint, available on Research Square: https://www.researchsquare.com/article/rs-3262987/v1, 2023.
  30. S. Rajappa, H. H. Bülthoff, and P. Stegagno, “Design and implementation of a novel architecture for physical human-UAV interaction,” The International Journal of Robotics Research, vol. 36, no. 5-7, pp. 800–819, May 2017.
  31. A. Briod, P. Kornatowski, A. Klaptocz, A. Garnier, M. Pagnamenta, J.-C. Zufferey, and D. Floreano, “Contact-based navigation for an autonomous flying robot,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 3987–3992.
  32. C. Papachristos, S. Khattak, and K. Alexis, “Haptic feedback-based reactive navigation for aerial robots subject to localization failure,” in 2019 IEEE Aerospace Conference, 2019, pp. 1–7.
  33. K. Shimonomura, “Tactile image sensors employing camera: A review,” Sensors, vol. 19, no. 18, 2019.
  34. S. Zhang, Z. Chen, Y. Gao, W. Wan, J. Shan, H. Xue, F. Sun, Y. Yang, and B. Fang, “Hardware technology of vision-based tactile sensor: A review,” IEEE Sensors Journal, vol. 22, no. 22, pp. 21 410–21 427, 2022.
  35. Y. Zhang, Z. Kan, Y. Yang, Y. A. Tse, and M. Y. Wang, “Effective estimation of contact force and torque for vision-based tactile sensors with helmholtz–hodge decomposition,” IEEE Robotics and Automation Letters, vol. 4, pp. 4094–4101, 2019.
  36. C. Sferrazza and R. D’Andrea, “Design, motivation and evaluation of a full-resolution optical tactile sensor,” Sensors, vol. 19, no. 4, p. 928, Feb. 2019. [Online]. Available: https://www.mdpi.com/1424-8220/19/4/928
  37. W. Yuan, S. Dong, and E. H. Adelson, “Gelsight: High-resolution robot tactile sensors for estimating geometry and force,” Sensors, vol. 17, no. 12, p. 2762, Nov. 2017.
  38. S. Luo, W. Yuan, E. Adelson, A. G. Cohn, and R. Fuentes, “Vitac: Feature sharing between vision and tactile sensing for cloth texture recognition,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 2722–2727.
  39. X. Guo, G. He, M. Mousaei, J. Geng, G. Shi, and S. Scherer, “Aerial interaction with tactile sensing,” in Preprint, available on ArXiv: https://arxiv.org/abs/2310.00142, 2023.
  40. E. Donlon, S. Dong, M. Liu, J. Li, E. Adelson, and A. Rodriguez, “Gelslim: A high-resolution, compact, robust, and calibrated tactile-sensing finger,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).   Madrid, Spain: IEEE, oct 2018, pp. 1927–1934.
  41. M. Lambeta, P.-W. Chou, S. Tian, B. Yang, B. Maloon, V. R. Most, D. Stroud, R. Santos, A. Byagowi, G. Kammerer, D. Jayaraman, and R. Calandra, “Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation,” IEEE Robotics and Automation Letters, vol. 5, pp. 3838–3845, 2020.
  42. B. Ward-Cherrier, N. Pestell, L. Cramphorn, B. Winstone, M. E. Giannaccini, J. Rossiter, , and N. F. Lepora, “The tactip family: Soft optical tactile sensors with 3d-printed biomimetic morphologies,” Soft Robotics, pp. 3216–227, 2018.
  43. X. Lin and M. Wiertlewski, “Sensing the frictional state of a robotic skin via subtractive color mixing,” IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2386–2392, Jul. 2019.
  44. A. C. Abad and A. Ranasinghe, “Low-cost gelsight with uv markings: Feature extraction of objects using alexnet and optical flow without 3d image reconstruction,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   Paris, France: IEEE, May 2020, pp. 3680–3685.
  45. W. Kim, W. D. Kim, J.-J. Kim, C.-H. Kim, and J. Kim, “Uvtac: Switchable uv marker-based tactile sensing finger for effective force estimation and object localization,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 6036–6043, Jul. 2022.
  46. L. Van Duong and V. A. Ho, “Large-scale vision-based tactile sensing for robot links: Design, modeling, and evaluation,” IEEE Transactions on Robotics, vol. 37, no. 2, pp. 390–403, 2021.
  47. Q. K. Luu, N. H. Nguyen, and V. A. Ho, “Simulation, learning, and application of vision-based tactile sensing at large scale,” IEEE Transactions on Robotics, vol. PP, pp. 1–17, 2023.
  48. A. Breuss, C. Sferrazza, J. Pleisch, R. D’Andrea, and R. Riener, “Unobtrusive sleep position classification using a novel optical tactile sensor,” in 2023 45th Annual International Conference of the IEEE Engineering in Medicine &\&& Biology Society (EMBC), 2023, pp. 1–5.
  49. H. Bhatia, V. Pascucci, and P.-T. Bremer, “The natural helmholtz-hodge decomposition for open-boundary flow analysis,” IEEE Transactions on Visualization and Computer Graphics, vol. 20, pp. 1566–1578, 2014.
  50. T. Kroeger, R. Timofte, D. Dai, and L. Van Gool, “Fast optical flow using dense inverse search,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14.   Springer, 2016, pp. 471–488.
  51. Q. Guo, M. K. Mandal, and M. Y. Li, “Efficient hodge–helmholtz decomposition of motion fields,” Pattern Recognition Letters, vol. 26, no. 4, pp. 493–501, Mar. 2005.
  52. C. Sferrazza, A. Wahlsten, C. Trueeb, and R. D’Andrea, “Ground truth force distribution for learning-based tactile sensing: A finite element approach,” IEEE Access, vol. 7, pp. 173 438–173 449, 2019.
  53. C. Sferrazza and R. D’Andrea, “Sim-to-real for high-resolution optical tactile sensing: From images to three-dimensional contact force distributions,” Soft Robotics, vol. 9, no. 5, pp. 926–937, 2022.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up