Probabilistic Foundations for Metacognition via Hybrid-AI (2502.05398v2)

Abstract: Metacognition is the concept of reasoning about an agent's own internal processes, and it has recently received renewed attention with respect to AI and, more specifically, machine learning systems. This paper reviews a hybrid-AI approach known as "error detecting and correcting rules" (EDCR) that allows for the learning of rules to correct perceptual (e.g., neural) models. Additionally, we introduce a probabilistic framework that adds rigor to prior empirical studies, and we use this framework to prove results on necessary and sufficient conditions for metacognitive improvement, as well as limits to the approach. A set of future