A Quantitative Autonomy Quantification Framework for Fully Autonomous Robotic Systems (2311.01939v2)
Abstract: Although autonomous functioning facilitates deployment of robotic systems in domains that admit limited human oversight on our planet and beyond, finding correspondence between task requirements and autonomous capability is still an open challenge. Consequently, a number of methods for quantifying autonomy have been proposed over the last three decades, but to our knowledge all these have no discernment of sub-mode features of variation of autonomy and some are based on metrics that violet the Goodhart's law. This paper focuses on the full autonomous mode and proposes a quantitative autonomy assessment framework based on task requirements. The framework starts by establishing robot task characteristics from which three autonomy metrics, namely requisite capability set, reliability and responsiveness are derived. These characteristics were founded on the realization that robots ultimately replace human skilled workers, from which a relationship between human job and robot task characteristics was established. Additionally, mathematical functions mapping metrics to autonomy as a two-part measure, namely of level and degree of autonomy are also presented. The distinction between level and degree of autonomy stemmed from the acknowledgment that autonomy is not just a question of existence, but also one of performance of requisite capability. The framework has been demonstrated on two case studies, namely autonomous vehicle at an on-road dynamic driving task and the DARPA subterranean challenge rules analysis. The framework provides not only a tool for quantifying autonomy, but also a regulatory interface and common language for autonomous systems developers and users. Its greatest feature is the ability to monitor system integrity when implemented online.
- S. Singh, “Critical reasons for crashes investigated in the national motor vehicle crash causation survey,” DOT HS 812 115. Washington, DC: National Highway Traffic Safety Administration (NHTSA), US Department of Transportation, Tech. Rep., 2015.
- W. Rankin, “Meda investigation process,” Boeing Commercial Aero, 2007.
- B. T. Clough, “Metrics, schmetrics! how the heck do you determine a uav’s autonomy anyway,” Air Force Research Lab Wright-Patterson AFB OH, Tech. Rep., 2002.
- E. Jackson, O. Williams, and K. Buchan, “Achieving robot autonomy,” in Defence Research Establishment Suffield, Proceedings of the 3 rd Conference on Military Robotic Applications p 242-248(SEE N 94-30275 08-37), 1991.
- A. Lampe and R. Chatila, “Performance measure for the evaluation of mobile robot autonomy,” in Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006. IEEE, 2006, pp. 4057–4062.
- C. C. Insaurralde and D. M. Lane, “Autonomy-assessment criteria for underwater vehicles,” in 2012 IEEE/OES Autonomous Underwater Vehicles (AUV). IEEE, 2012, pp. 1–8.
- F. Kendoul, “Towards a unified framework for uas autonomy and technology readiness assessment (atra),” in Autonomous Control Systems and Vehicles. Springer, 2013, pp. 55–71.
- R. Clothier, B. Williams, and T. Perez, “A review of the concept of autonomy in the context of the safety regulation of civil unmanned aircraft systems,” in Proceedings of the Australian System Safety Conference 2013 (ASSC 2013)[Conferences in Research and Practice in Information Technology (CRPIT), Conferences in Research and Practice in Information Technology]. Australian Computer Society Inc., 2014, pp. 15–27.
- M. Elbanhawi, A. Mohamed, R. Clothier, J. Palmer, M. Simic, and S. Watkins, “Enabling technologies for autonomous mav operations,” Progress in Aerospace Sciences, vol. 91, pp. 27–52, 2017.
- H.-M. Huang, E. Messina, and J. Albus, “Autonomy levels for unmanned systems (alfus) framework, volume ii,” Gaithersburg, MD.
- S. J3016, “Surface vehicle recommended practice-taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles,” 2021.
- S. of California Department of Motor Vehicles, “California code of regulations title 13, division 1, chapter 1, article 3.7. testing of autonomous vehicles,” 2022. [Online]. Available: https://www.dmv.ca.gov/portal/file/adopted-regulatory-text-pdf/
- A. Doboli, D. Curiac, D. Pescaru, S. Doboli, W. Tang, C. Volosencu, M. Gilberti, O. Banias, and C. Istin, “Cities of the future: Employing wireless sensor networks for efficient decision making in complex environments,” arXiv preprint arXiv:1808.01169, 2018.
- T. B. Curtin, D. M. Crimmins, J. Curcio, M. Benjamin, and C. Roper, “Autonomous underwater vehicles: trends and transformations,” Marine Technology Society Journal, vol. 39, no. 3, pp. 65–75, 2005.
- B. Hasslacher and M. W. Tilden, “Living machines,” Robotics and autonomous systems, vol. 15, no. 1-2, pp. 143–169, 1995.
- G. N. Saridis, “An analytic formulation of knowledge based systems for intelligent machines,” Proc. NATO ARW on Knowledge-Based Robot Control, Bonas, France (GN Saridis andH. Stephanou, eds.). Springer, New York, 1989.
- J. ICAO, “International standards and recommended practices,” in Annex 10 to the Convention on International Civil Aviation, vol. 1, 1996.
- I. Annex, “Vol. 1,” annex 10 to convention on international civil aviation: Aeronautical telecommunications,” Volume I-Radio Navigation Aids”, Sixth Edition, ICAO Montreal, 2006.
- D. Paz, P.-j. Lai, N. Chan, Y. Jiang, and H. I. Christensen, “Autonomous vehicle benchmarking using unbiased metrics,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 6223–6228.
- T. G. Reid, S. E. Houts, R. Cammarata, G. Mills, S. Agarwal, A. Vora, and G. Pandey, “Localization requirements for autonomous vehicles,” arXiv preprint arXiv:1906.01061, 2019.
- “Darpa subterranean challenge: Competition rules final event.” [Online]. Available: https://www.subtchallenge.com/resources.html
- Nasser Gyagenda (1 paper)
- Hubert Roth (2 papers)