A Precision Drone Landing System using Visual and IR Fiducial Markers and a Multi-Payload Camera (2403.03806v1)
Abstract: We propose a method for autonomous precision drone landing with fiducial markers and a gimbal-mounted, multi-payload camera with wide-angle, zoom, and IR sensors. The method has minimal data requirements; it depends primarily on the direction from the drone to the landing pad, enabling it to switch dynamically between the camera's different sensors and zoom factors, and minimizing auxiliary sensor requirements. It eliminates the need for data such as altitude above ground level, straight-line distance to the landing pad, fiducial marker size, and 6 DoF marker pose (of which the orientation is problematic). We leverage the zoom and wide-angle cameras, as well as visual April Tag fiducial markers to conduct successful precision landings from much longer distances than in previous work (168m horizontal distance, 102m altitude). We use two types of April Tags in the IR spectrum - active and passive - for precision landing both at daytime and nighttime, instead of simple IR beacons used in most previous work. The active IR landing pad is heated; the novel, passive one is unpowered, at ambient temperature, and depends on its high reflectivity and an IR differential between the ground and the sky. Finally, we propose a high-level control policy to manage initial search for the landing pad and subsequent searches if it is lost - not addressed in previous work. The method demonstrates successful landings with the landing skids at least touching the landing pad, achieving an average error of 0.19m. It also demonstrates successful recovery and landing when the landing pad is temporarily obscured.
- H. Kim, C.-U. Hyun, H.-D. Park, and J. Cha, “Image Mapping Accuracy Evaluation Using UAV with Standalone, Differential (RTK), and PPP GNSS Positioning Techniques in an Abandoned Mine Site,” Sensors, vol. 23, no. 13, 2023. [Online]. Available: https://www.mdpi.com/1424-8220/23/13/5858
- M. Krogius, A. Haggenmiller, and E. Olson, “Flexible Layouts for Fiducial Tags,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE Press, 2019, p. 1898–1903. [Online]. Available: https://doi.org/10.1109/IROS40897.2019.8967787
- M. Fiala, “ARTag, a fiducial marker system using digital techniques,” 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 590–596 vol. 2, 2005. [Online]. Available: https://api.semanticscholar.org/CorpusID:5746274
- S. Garrido-Jurado, R. Muñoz-Salinas, F. Madrid-Cuevas, and M. Marín-Jiménez, “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognition, vol. 47, no. 6, pp. 2280–2292, 2014. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0031320314000235
- P. Lightbody, T. Krajník, and M. Hanheide, “A versatile high-performance visual fiducial marker detection system with scalable identity encoding,” Proceedings of the Symposium on Applied Computing, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:5013993
- G. Schweighofer and A. Pinz, “Robust Pose Estimation from a Planar Target,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2024–2030, 2006.
- J. Springer and M. Kyas, “Evaluation of Orientation Ambiguity and Detection Rate in April Tag and WhyCode,” in 2022 Sixth IEEE International Conference on Robotic Computing (IRC). Naples, Italy: IEEE Computer Society, Dec 2022, pp. 281–286. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/IRC55401.2022.00054
- O. Araar, N. Aouf, and I. Vitanov, “Vision based autonomous landing of multirotor uav on moving platform,” Journal of Intelligent & Robotic Systems, vol. 85, pp. 369–384, 2016.
- M. A. Olivares-Mendez, I. F. Mondragón, and P. Campoy, “Autonomous Landing of an Unmanned Aerial Vehicle using Image-Based Fuzzy Control,” IFAC Proceedings Volumes, vol. 46, no. 30, pp. 79–86, 2013, 2nd IFAC Workshop on Research, Education and Development of Unmanned Aerial Systems. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1474667015402757
- G. Badakis, M. Koutsoubelias, and S. Lalis, “Robust Precision Landing for Autonomous Drones Combining Vision-based and Infrared Sensors,” in 2021 IEEE Sensors Applications Symposium (SAS), 2021, pp. 1–6.
- E. Nowak, K. Gupta, and H. Najjaran, “Development of a Plug-and-Play Infrared Landing System for Multirotor Unmanned Aerial Vehicles,” in 2017 14th Conference on Computer and Robot Vision (CRV), 2017, pp. 256–260.
- N. Xuan-Mung, S. K. Hong, N. P. Nguyen, L. N. N. T. Ha, and T.-L. Le, “Autonomous Quadcopter Precision Landing Onto a Heaving Platform: New Method and Experiment,” IEEE Access, vol. 8, pp. 167 192–167 202, 2020.
- K. Pluckter and S. Scherer, “Precision UAV Landing in Unstructured Environments,” in Proceedings of the 2018 International Symposium on Experimental Robotics, ser. Springer Proceedings in Advanced Robotics, J. Xiao, T. Kröger, and O. Khatib, Eds., vol. 11. Cham: Springer, 2018, pp. 177––187.
- R. Polvara, S. Sharma, J. Wan, A. Manning, and R. Sutton, “Towards autonomous landing on a moving vessel through fiducial markers,” in 2017 European Conference on Mobile Robots (ECMR), 2017, pp. 1–6.
- D. Falanga, A. Zanchettin, A. Simovic, J. Delmerico, and D. Scaramuzza, “Vision-based autonomous quadrotor landing on a moving platform,” in 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), 2017, pp. 200–207.
- A. Borowczyk, D.-T. Nguyen, A. Phu-Van Nguyen, D. Q. Nguyen, D. Saussié, and J. L. Ny, “Autonomous Landing of a Multirotor Micro Air Vehicle on a High Velocity Ground Vehicle**This work was partially supported by CFI JELF Award 32848 and a hardware donation from DJI.” IFAC-PapersOnLine, vol. 50, no. 1, pp. 10 488–10 494, 2017, 20th IFAC World Congress. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2405896317326186
- G. Cho, J. Choi, G. Bae, and H. Oh, “Autonomous ship deck landing of a quadrotor UAV using feed-forward image-based visual servoing,” Aerospace Science and Technology, vol. 130, p. 107869, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1270963822005430
- T. Jiang, D. Lin, and T. Song, “Vision-based autonomous landing of a quadrotor using a gimbaled camera,” Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, vol. 233, no. 14, pp. 5093–5106, 2019. [Online]. Available: https://doi.org/10.1177/0954410019837777
- M. Demirhan and C. Premachandra, “Development of an Automated Camera-Based Drone Landing System,” IEEE Access, vol. 8, pp. 202 111–202 121, 2020.
- J. Lim, T. Lee, S. Pyo, J. Lee, J. Kim, and J. Lee, “Hemispherical InfraRed (IR) Marker for Reliable Detection for Autonomous Landing on a Moving Ground Vehicle From Various Altitude Angles,” IEEE/ASME Transactions on Mechatronics, vol. 27, no. 1, pp. 485–492, 2022.
- H. Tanaka, K. Ogata, and Y. Matsumoto, “Solving pose ambiguity of planar visual marker by wavelike two-tone patterns,” 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 568–573, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:30231919
- H. Tanaka and Y. Matsumoto, “Autonomous Drone Guidance and Landing System Using AR/high-accuracy Hybrid Markers,” in 2019 IEEE 8th Global Conference on Consumer Electronics (GCCE), 2019, pp. 598–599.
- J. Springer and M. Kyas, “Autonomous Drone Landing with Fiducial Markers and a Gimbal-Mounted Camera for Active Tracking,” in 2022 Sixth IEEE International Conference on Robotic Computing (IRC). Naples, Italy: IEEE Computer Society, Dec 2022, pp. 243–247. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/IRC55401.2022.00047
- M. D. Dogan, A. Taka, M. Lu, Y. Zhu, A. Kumar, A. Gupta, and S. Mueller, “InfraredTags: Embedding Invisible AR Markers and Barcodes Using Low-Cost, Infrared-Based 3D Printing and Imaging Tools,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, ser. CHI ’22. New York, NY, USA: Association for Computing Machinery, 2022. [Online]. Available: https://doi.org/10.1145/3491102.3501951
- S. Khattak, C. Papachristos, and K. Alexis, “Marker Based Thermal-Inertial Localization for Aerial Robots in Obscurant Filled Environments,” 11 2018, pp. 565–575.
- R. M. Claro, D. B. Silva, and A. M. Pinto, “ArTuga: A novel multimodal fiducial marker for aerial robotics,” Robotics and Autonomous Systems, vol. 163, p. 104398, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0921889023000374
- Z. M. Zhang and G. Machin, “Chapter 1 Overview of Radiation Thermometry,” in Radiometric Temperature Measurements: I. Fundamentals, ser. Experimental Methods in the Physical Sciences, Z. M. Zhang, B. K. Tsai, and G. Machin, Eds. Academic Press, 2009, vol. 42, pp. 22–23,107. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1079404209042015
- Matrice 350 RTK User Manual. Da Jiang Innovations (DJI), 2023. [Online]. Available: https://dl.djicdn.com/downloads/matrice_350_rtk/Matrice_350_RTK_User_Manual_en.pdf
- Zenmuse H20 Series User Manual. Da Jiang Innovations (DJI), May 2020. [Online]. Available: {https://dl.djicdn.com/downloads/Zenmuse_H20_Series/Zenmuse_H20_Series_User_Manual-EN.pdf}
- J. Springer, “Payload-SDK App: RPi Autonomous Landing,” https://github.com/uzgit/Payload-SDK/tree/apriltag, 2023.
- Matrice Fiducial Landings Demo. Vimeo, March 2024. [Online]. Available: {https://vimeo.com/j0shua/matrice-fiducial-demo-visir}