Confidence-Aware Decision-Making and Control for Tool Selection (2403.03808v1)
Abstract: Self-reflecting about our performance (e.g., how confident we are) before doing a task is essential for decision making, such as selecting the most suitable tool or choosing the best route to drive. While this form of awareness -- thinking about our performance or metacognitive performance -- is well-known in humans, robots still lack this cognitive ability. This reflective monitoring can enhance their embodied decision power, robustness and safety. Here, we take a step in this direction by introducing a mathematical framework that allows robots to use their control self-confidence to make better-informed decisions. We derive a mathematical closed-form expression for control confidence for dynamic systems (i.e., the posterior inverse covariance of the control action). This control confidence seamlessly integrates within an objective function for decision making, that balances the: i) performance for task completion, ii) control effort, and iii) self-confidence. To evaluate our theoretical account, we framed the decision-making within the tool selection problem, where the agent has to select the best robot arm for a particular control task. The statistical analysis of the numerical simulations with randomized 2DOF arms shows that using control confidence during tool selection improves both real task performance, and the reliability of the tool for performance under unmodelled perturbations (e.g., external forces). Furthermore, our results indicate that control confidence is an early indicator of performance and thus, it can be used as a heuristic for making decisions when computation power is restricted or decision-making is intractable. Overall, we show the advantages of using confidence-aware decision-making and control scheme for dynamic systems.
- N. Yeung and C. Summerfield, “Metacognition in human decision-making: confidence and error monitoring,” Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 367, no. 1594, pp. 1310–1321, 2012.
- F. Bontje, “Are you sure? modelling the confidence of a driver in left-turn gap acceptance decisions,” TU Delft, 2023.
- S. M. Fleming, R. J. Dolan, and C. D. Frith, “Metacognition: computation, biology and function,” pp. 1280–1286, 2012.
- S. M. Fleming and N. D. Daw, “Self-evaluation of decision-making: A general bayesian framework for metacognitive computation.” Psychological review, vol. 124, no. 1, p. 91, 2017.
- C. Parkan and M.-L. Wu, “Decision-making and performance measurement models with applications to robot selection,” Computers & Industrial Engineering, vol. 36, no. 3, pp. 503–523, 1999.
- H. Saeidi, J. D. Opfermann, M. Kam, S. Raghunathan, S. Léonard, and A. Krieger, “A confidence-based shared control strategy for the smart tissue autonomous robot (star),” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 1268–1275.
- H. Saeidi and Y. Wang, “Trust and self-confidence based autonomy allocation for robotic systems,” in 2015 54th IEEE Conference on Decision and Control (CDC). IEEE, 2015, pp. 6052–6057.
- D. Fridovich-Keil, A. Bajcsy, J. F. Fisac, S. L. Herbert, S. Wang, A. D. Dragan, and C. J. Tomlin, “Confidence-aware motion prediction for real-time collision avoidance,” The International Journal of Robotics Research, vol. 39, no. 2-3, pp. 250–265, 2020.
- Z. Cao, S. Xu, H. Peng, D. Yang, and R. Zidek, “Confidence-aware reinforcement learning for self-driving cars,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 7419–7430, 2021.
- J. Moon, J. Kim, Y. Shin, and S. Hwang, “Confidence-aware learning for deep neural networks,” in international conference on machine learning. PMLR, 2020, pp. 7034–7044.
- G. Papadopoulos, P. J. Edwards, and A. F. Murray, “Confidence estimation methods for neural networks: A practical comparison,” IEEE transactions on neural networks, vol. 12, no. 6, pp. 1278–1287, 2001.
- A. Lederer, Q. Hao, and S. Hirche, “Confidence regions for simulations with learned probabilistic models,” in 2020 American Control Conference (ACC). IEEE, 2020, pp. 3947–3952.
- W. M. Kouw, “Information-seeking polynomial narx model-predictive control through expected free energy minimization,” IEEE Control Systems Letters, 2023.
- C. Hesp, R. Smith, T. Parr, M. Allen, K. J. Friston, and M. J. Ramstead, “Deeply felt affect: The emergence of valence in deep active inference,” Neural computation, vol. 33, no. 2, pp. 398–446, 2021.
- K. Friston, “The free-energy principle: a unified brain theory?” Nature reviews neuroscience, vol. 11, no. 2, pp. 127–138, 2010.
- P. Lanillos, C. Meo, C. Pezzato, A. A. Meera, M. Baioumy, W. Ohata, A. Tschantz, B. Millidge, M. Wisse, C. L. Buckley et al., “Active inference in robotics and artificial agents: Survey and challenges,” arXiv preprint arXiv:2112.01871, 2021.
- A. Anil Meera and P. Lanillos, “Towards metacognitive robot decision making for tool selection,” in International Workshop on Active Inference. Springer, 2023, pp. 31–42.
- G. Oliver, P. Lanillos, and G. Cheng, “An empirical study of active inference on a humanoid robot,” IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 2, pp. 462–471, 2021.
- A. Anil Meera and M. Wisse, “Dynamic expectation maximization algorithm for estimation of linear systems with colored noise,” Entropy, vol. 23, no. 10, p. 1306, 2021.
- K. J. Friston, N. Trujillo-Barreto, and J. Daunizeau, “Dem: a variational treatment of dynamic systems,” Neuroimage, vol. 41, no. 3, pp. 849–885, 2008.