Model-free, Model-based, and General Intelligence (1806.02308v1)

Abstract: During the 60s and 70s, AI researchers explored intuitions about intelligence by writing programs that displayed intelligent behavior. Many good ideas came out from this work but programs written by hand were not robust or general. After the 80s, research increasingly shifted to the development of learners capable of inferring behavior and functions from experience and data, and solvers capable of tackling well-defined but intractable models like SAT, classical planning, Bayesian networks, and POMDPs. The learning approach has achieved considerable success but results in black boxes that do not have the flexibility, transparency, and generality of their model-based counterparts. Model-based approaches, on the other hand, require models and scalable algorithms. Model-free learners and model-based solvers have close parallels with Systems 1 and 2 in current theories of the human mind: the first, a fast, opaque, and inflexible intuitive mind; the second, a slow, transparent, and flexible analytical mind. In this paper, I review developments in AI and draw on these theories to discuss the gap between model-free learners and model-based solvers, a gap that needs to be bridged in order to have intelligent systems that are robust and general.

• The paper's main contribution is bridging model-free, data-driven learning with transparent model-based solving to advance AI systems.
• It details the historical evolution of AI from early symbolic programming to modern machine learning and planning methods.
• The study advocates integrating both paradigms to overcome limitations and achieve robust, interpretable general intelligence.

Model-free, Model-based, and General Intelligence

The paper "Model-free, Model-based, and General Intelligence" by Hector Geffner presents an incisive analysis of contemporary AI research trajectories, examining the longstanding dichotomy between model-free learners and model-based solvers. The author provides historical context for the evolution of AI, highlighting transitions from handcrafted intelligent programs to data-driven learning systems, emphasizing the parallel development of both paradigms.

Historical Context and Evolution

Geffner traces the roots of AI back to the work of Alan Turing and the early days when AI research predominantly involved programming intelligent behavior directly. This period, marked by the emphasis on representations and knowledge within programs, saw significant contributions but also illustrated the limitations in program robustness and generality. By the 1980s, the field shifted towards algorithms solving well-defined models as a response to these limitations, birthing the era of learners and solvers.

Model-Free Learners vs. Model-Based Solvers

Model-free learners, exemplified by deep learning (DL) and deep reinforcement learning (DRL), derive their utility from processing vast quantities of data to acquire a function that can generalize across specific tasks. These methods have achieved significant breakthroughs in various domains; however, they function primarily as complex black boxes lacking interpretability and flexibility. On the other hand, model-based solvers, which automatically generate solutions from models, offer transparency and robustness. These solvers, though reliant on pre-defined models, can address a wide array of problems without the need for extensive training data, reflecting the characteristics of System 2 processes in cognitive psychology.

Bridging Learners and Solvers

Central to the paper is the argument for bridging the conceptual and operational gaps between learners and solvers. A notable contribution of Geffner is the exploration of how both paradigms can benefit from integration, similar to the cognitive interplay between human Systems 1 and 2. AlphaZero, a notable exemplar, merges learning with search-based methods, iteratively improving its policy via self-play, mirroring aspects of model-based reasoning.

Implications and Future Directions

Geffner articulates the potential limits of current DRL approaches due to their fixed input size constraint, limiting their ability to handle arbitrary instances in the same manner as solvers. He highlights key areas for advancing AI capabilities:

• Model Learning: The development of techniques to autonomously derive models from observations and actions is essential for enhancing the flexibility of AI systems.
• Variable and Feature Learning: Identifying and abstracting the relevant variables, akin to the human ability to discern essential features in diverse contexts, remains an open challenge.
• Generalized Planning: Efforts in DRL and planning are converging in their pursuit of general policies applicable across diverse problem instances.

In conclusion, the paper underscores the necessity for future AI systems to amalgamate the strengths of both model-free and model-based paradigms, leading to more robust and flexible intelligence. Such integration holds potential not only for addressing complex computational tasks but also for enhancing interpretability and accountability in AI systems. Future research should further explore these synergies, providing insights that could fundamentally reshape the development of general intelligence in artificial systems.