Race Against the Machine: a Fully-annotated, Open-design Dataset of Autonomous and Piloted High-speed Flight (2311.02667v2)
Abstract: Unmanned aerial vehicles, and multi-rotors in particular, can now perform dexterous tasks in impervious environments, from infrastructure monitoring to emergency deliveries. Autonomous drone racing has emerged as an ideal benchmark to develop and evaluate these capabilities. Its challenges include accurate and robust visual-inertial odometry during aggressive maneuvers, complex aerodynamics, and constrained computational resources. As researchers increasingly channel their efforts into it, they also need the tools to timely and equitably compare their results and advances. With this dataset, we want to (i) support the development of new methods and (ii) establish quantitative comparisons for approaches originating from the broader robotics and artificial intelligence communities. We want to provide a one-stop resource that is comprehensive of (i) aggressive autonomous and piloted flight, (ii) high-resolution, high-frequency visual, inertial, and motion capture data, (iii) commands and control inputs, (iv) multiple light settings, and (v) corner-level labeling of drone racing gates. We also release the complete specifications to recreate our flight platform, using commercial off-the-shelf components and the open-source flight controller Betaflight, to democratize drone racing research. Our dataset, open-source scripts, and drone design are available at: https://github.com/tii-racing/drone-racing-dataset
- E. Kaufmann, et al., “Champion-level drone racing using deep reinforcement learning,” Nature, vol. 620, no. 7976, pp. 982–987, Aug 2023.
- A. Romero, et al., “Model predictive contouring control for time-optimal quadrotor flight,” IEEE Transactions on Robotics, vol. 38, no. 6, pp. 3340–3356, 2022.
- J. Delmerico, et al., “Are we ready for autonomous drone racing? the uzh-fpv drone racing dataset,” in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 6713–6719.
- Y. Song, et al., “Reaching the limit in autonomous racing: Optimal control versus reinforcement learning,” Science Robotics, vol. 8, no. 82, 2023.
- J. Deng, et al., “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
- “AlphaPilot – Lockheed Martin AI Drone Racing Innovation Challenge,” https://www.herox.com/alphapilot/teams, 2019, [Online].
- A. Antonini, et al., “The blackbird dataset: A large-scale dataset for uav perception in aggressive flight,” in Proceedings of the 2018 International Symposium on Experimental Robotics, J. Xiao, et al., Eds. Cham: Springer International Publishing, 2020, pp. 130–139.
- M. Burri, et al., “The EuRoC micro aerial vehicle datasets,” The International Journal of Robotics Research, vol. 35, no. 10, pp. 1157–1163, 2016.
- K. Sun, et al., “Robust stereo visual inertial odometry for fast autonomous flight,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 965–972, 2018.
- C. Pfeiffer et al., “Human-piloted drone racing: Visual processing and control,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3467–3474, 2021.
- L. Bauersfeld, et al., “NeuroBEM: Hybrid aerodynamic quadrotor model,” in Robotics: Science and Systems XVII. Robotics: Science and Systems Foundation, jul 2021.
- P. Newman et al., “Data papers—peer reviewed publication of high quality data sets,” pp. 587–587, 2009.
- A. Geiger, et al., “Vision meets robotics: The kitti dataset,” International Journal of Robotics Research (IJRR), 2013.
- W. Zhao, et al., “UTIL: An ultra-wideband time-difference-of-arrival indoor localization dataset,” 2022.
- D. Hanover, et al., “Autonomous drone racing: A survey,” 2023.
- A. Loquercio, et al., “Learning high-speed flight in the wild,” Science Robotics, vol. 6, no. 59, 2021.
- P. Foehn, et al., “Alphapilot: autonomous drone racing,” Autonomous Robots, vol. 46, no. 1, pp. 307–320, Jan 2022.
- A. L. Majdik, et al., “The Zurich urban micro aerial vehicle dataset,” The International Journal of Robotics Research, vol. 36, no. 3, pp. 269–273, 2017.
- P. Foehn, et al., “Agilicious: Open-source and open-hardware agile quadrotor for vision-based flight,” Science Robotics, vol. 7, no. 67, p. eabl6259, 2022.
- C. de Wagter, et al., “The artificial intelligence behind the winning entry to the 2019 ai robotic racing competition,” ArXiv, vol. abs/2109.14985, 2021.
- E. Kaufmann, et al., “Beauty and the beast: Optimal methods meet learning for drone racing,” 2019 International Conference on Robotics and Automation (ICRA), pp. 690–696, 2018.
- MultiWii, “Multiwii serial protocol,” http://www.multiwii.com/wiki/index.php?title=Multiwii˙Serial˙Protocol.
- Betaflight, “The betaflight open source flight controller firmware project,” https://github.com/betaflight/betaflight.
- MultiGP, “Multigp drone racing gate,” https://www.multigp.com/product/drone-racing-gate-bundle/.
- M. Faessler, et al., “Differential flatness of quadrotor dynamics subject to rotor drag for accurate tracking of high-speed trajectories,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 620–626, 2018.
- D. Falanga, et al., “Pampc: Perception-aware model predictive control for quadrotors,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 1–8.
- H. X. Pham, et al., “Deep learning for vision-based navigation in autonomous drone racing,” Deep Learning for Robot Perception and Cognition, 2022.
- K. Chen, et al., “Mmdetection: Open mmlab detection toolbox and benchmark,” 2019.
- M. Contributors, “Openmmlab pose estimation toolbox and benchmark,” https://github.com/open-mmlab/mmpose, 2020.
- “COCO - Common Objects in Context - Data format,” [Accessed 30. Aug. 2023]. [Online]. Available: https://cocodataset.org/#format-data
- K. Shoemake, “Animating rotation with quaternion curves,” in Proceedings of the 12th Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’85. New York, NY, USA: Association for Computing Machinery, 1985, p. 245–254.