Thinking Fast and Slow in AI: the Role of Metacognition
(2110.01834v1)
Published 5 Oct 2021 in cs.AI
Abstract: AI systems have seen dramatic advancement in recent years, bringing many applications that pervade our everyday life. However, we are still mostly seeing instances of narrow AI: many of these recent developments are typically focused on a very limited set of competencies and goals, e.g., image interpretation, natural language processing, classification, prediction, and many others. Moreover, while these successes can be accredited to improved algorithms and techniques, they are also tightly linked to the availability of huge datasets and computational power. State-of-the-art AI still lacks many capabilities that would naturally be included in a notion of (human) intelligence. We argue that a better study of the mechanisms that allow humans to have these capabilities can help us understand how to imbue AI systems with these competencies. We focus especially on D. Kahneman's theory of thinking fast and slow, and we propose a multi-agent AI architecture where incoming problems are solved by either system 1 (or "fast") agents, that react by exploiting only past experience, or by system 2 (or "slow") agents, that are deliberately activated when there is the need to reason and search for optimal solutions beyond what is expected from the system 1 agent. Both kinds of agents are supported by a model of the world, containing domain knowledge about the environment, and a model of "self", containing information about past actions of the system and solvers' skills.
The paper "Thinking Fast and Slow in AI: the Role of Metacognition" (Ganapini et al., 2021) proposes an AI architecture inspired by Daniel Kahneman's dual-system theory of human cognition. The core idea is to build AI systems that can exhibit more generalizability, adaptability, robustness, and efficiency by integrating fast, intuitive processing (System 1) with slow, deliberate reasoning (System 2), orchestrated by a metacognitive component.
The authors argue that current state-of-the-art AI systems, while successful in narrow domains, lack many capabilities associated with human intelligence. This is attributed partly to their reliance on either purely data-driven, fast methods or explicit, symbolic reasoning, without seamless integration and adaptive selection.
To address this, the paper introduces the SOFAI (SlOw and Fast AI) architecture. This is a multi-agent system consisting of:
System 1 (S1) Solvers: These agents are designed for "fast thinking." They react quickly, ideally in constant time relative to the problem size, relying primarily on past experience stored in a "model of self." S1 solvers provide a proposed solution and a confidence level. They are analogous to human intuition and heuristics.
System 2 (S2) Solvers: These agents are for "slow thinking." They are activated when the problem requires deliberate reasoning, complex calculation, or searching for optimal solutions. S2 solvers are typically more resource-intensive (time, memory) than S1 solvers. They are analogous to human logical thinking and deliberation.
Model of the World: Contains domain knowledge about the environment, expected tasks, and available actions.
Model of Self: Stores information about the system's past actions, performance of S1 and S2 solvers on specific tasks (resource consumption, rewards), available system resources, expected rewards for solving tasks, and past metacognitive agent costs.
Model of Others: Holds knowledge and beliefs about other agents in the environment (though less detailed in this paper).
Model Updater Agent: Runs in the background to keep the models of the world, self, and others up-to-date based on new information, decisions, and outcomes.
Meta-cognitive (MC) Agent: This is the central component responsible for monitoring and controlling the problem-solving process. Its primary role is arbitration – deciding whether to accept the solution proposed by an S1 solver or to invoke an S2 solver instead.
Implementation of the Meta-cognitive Agent:
The MC agent's decision-making process is crucial and is designed to be resource-conscious, mimicking the human process of rapid assessment followed by potentially deeper deliberation. It operates in two phases:
MC1 (Fast/Approximate Phase):
Resource Check: Quickly assesses if there are sufficient resources (time, memory) available to complete both MC1 and a potential MC2 phase. If resources are insufficient, it immediately accepts the S1 solution.
Confidence vs. Reward Check: Compares the confidence level provided by the S1 solver for its proposed solution against the expected reward for successfully solving the task (retrieved from the Model of self or world). If the S1 confidence is high enough relative to the expected reward, MC1 accepts the S1 solution.
Activation of MC2: If resources are sufficient and the S1 confidence is not high enough relative to the expected reward, MC1 proceeds to activate the MC2 phase for a more careful decision.
This phase serves as a rapid filter, allowing the system to quickly adopt the S1 solution when it's likely good enough or when resources are constrained, saving the cost of more complex metacognition.
MC2 (Slow/Deliberate Phase):
Activated by MC1 when a more careful assessment is needed.
Evaluate S2: Evaluates the expected reward of using an appropriate S2 solver for the current task in the current state. This evaluation uses information from the Model of self about the S2 solver's past performance (rewards, costs) and the expected cost of running it. The paper mentions approximating forward analysis here to avoid excessive cost.
Evaluate S1 Solution: Evaluates the expected reward of accepting the action proposed by the S1 solver. This might involve retrieving expected immediate and future rewards for that action from the Model of self.
Arbitration: Compares the expected additional reward of running the S2 solver (compared to using the S1 solution) against the expected cost of running the S2 solver.
If (Expected S2 Reward - Expected S1 Solution Reward) > Expected S2 Cost, MC2 activates the S2 solver.
Otherwise, MC2 accepts the S1 solver's solution.
The rationale for MC2's decision is to only incur the higher cost of S2 reasoning if the expected gain in solution quality is substantial enough to justify that cost.
The architecture follows an S1-by-default strategy: S1 solvers process incoming problems immediately, and the MC agent decides after getting an S1 output whether S2 is necessary. This minimizes latency when simple problems can be handled by S1 and allows the MC agent to use S1's output and confidence as inputs to its decision.
Implementation Considerations and Applications:
The paper highlights that the architecture is being explored in sequential decision problems. The MC agent can be applied:
At Each Decision Step: Where solvers determine a single move or action, and the MC decides which solver (S1 or S2) makes each subsequent decision. This offers maximum flexibility.
Once for a Sequence: Where solvers produce an entire plan or trajectory. The MC decides upfront which solver generates the full sequence. This might be suitable when solvers have deep domain knowledge.
Two specific instances are under development:
Constrained Grid Pathfinding: Solvers determine single moves on a grid with various constraints. The MC agent operates at each step to choose the next move solver. This instance reflects the MC-at-each-decision-step approach (Glazier et al., 2021).
Epistemic Planning: Solvers are full planners capable of generating entire plans to achieve goals involving knowledge and beliefs. The MC agent could potentially operate once per planning problem, deciding which planner to use. This instance might incorporate problem/solver simplification steps within the MC process.
The authors note simplified assumptions in the current paper (e.g., one S1/S2 solver per task, perfect past experience matching) and mention ongoing work to handle multiple solvers, problem similarity, and different methods for combining confidence and value (currently using a risk-averse multiplication).
In summary, the SOFAI architecture provides a practical framework for building more sophisticated AI systems by integrating fast and slow processing modes, governed by a multi-phase metacognitive control mechanism that balances solution quality, confidence, resources, and expected rewards. Its implementation requires defining specific S1 and S2 solvers for a given domain, populating and updating the system's models, and implementing the two-phase decision logic of the MC agent based on quantifiable estimates of confidence, reward, and cost.