The Efficiency of Human Cognition Reflects Planned Information Processing

Abstract: Planning is useful. It lets people take actions that have desirable long-term consequences. But, planning is hard. It requires thinking about consequences, which consumes limited computational and cognitive resources. Thus, people should plan their actions, but they should also be smart about how they deploy resources used for planning their actions. Put another way, people should also "plan their plans". Here, we formulate this aspect of planning as a meta-reasoning problem and formalize it in terms of a recursive Bellman objective that incorporates both task rewards and information-theoretic planning costs. Our account makes quantitative predictions about how people should plan and meta-plan as a function of the overall structure of a task, which we test in two experiments with human participants. We find that people's reaction times reflect a planned use of information processing, consistent with our account. This formulation of planning to plan provides new insight into the function of hierarchical planning, state abstraction, and cognitive control in both humans and machines.

• The paper's main contribution is the formulation of human meta-planning as a recursive Bellman problem that quantifies planning costs using an information-theoretic approach.
• The methodology employs experiments in 2D Gridworld mazes, demonstrating that reaction times correlate with planned information-processing costs.
• This study implies that integrating hierarchical planning and state abstraction in AI requires mimicking human meta-planning strategies.

Planned Information Processing and Human Cognition

Introduction

This paper examines the efficiency of human cognition through a proposed framework of planned information processing. It explores the concept of planning actions and meta-planning—the planning of planning itself—by formalizing this as a meta-reasoning problem. By introducing a recursive Bellman objective that incorporates both task rewards and information-theoretic planning costs, a quantitative account of how humans should engage in planning and meta-planning is proposed. Two experiments are conducted to validate these theoretical predictions, and the implications are examined within the context of hierarchical planning, state abstraction, and cognitive control.

Meta-Reasoning and Planning

The authors begin by exploring how human decision-making often involves sketching partial plans based on relevancy, deferring the details to when necessary. They propose that this behavior stems from the high cost of representing detailed plans in terms of time and memory. This notion of planning one's plans is rigorously formalized via a recursive Bellman objective, which quantifies the planning process against information-theoretic costs. Thus, a structured task or problem is analyzed through both the framework of total rewards and the computational costs associated with planning this task.

Information-Theoretic Costs in Planning

The core of the proposed model involves integrating planning costs defined through an information-theoretic lens. By employing a Markov Decision Process (MDP) framework, the relationship between task structure, state transitions, and reward optimization is clearly laid out. The novel contribution is how this combines with planning costs captured as Kullback-Leibler divergences from a reference policy over simulated states. The recursive Bellman equation is extended to reflect these aspects of task planning, providing a comprehensive means of evaluating meta-planning decisions.

Experimental Validation of Meta-Planning

Experiment 1: Parametric Mazes

The first experiment tests this conceptual framework using 2D Gridworld mazes, evaluating reaction times (RTs) as a reflection of planning costs. The results demonstrated that participants' RTs were significantly predicted by the planned information-processing costs rather than solely by direct action costs. This supports the hypothesis that human decision-making encompasses planned information-processing strategies.

Experiment 2: Probing Partial Plans

The second experiment involved teleportation within a Gridworld to probe the inferred partial plans beneath participant behaviors. Post-teleportation RTs served to confirm if pre-conceived partial plans align with the theoretical model. Reactions supported the planned partial planning assertion, illustrating adaptability and the active role of meta-planning in cognitive strategies.

Implications for Cognitive Modeling and AI

The notion of meta-planning advanced in this study provides a more nuanced understanding of human cognition and its implementation in AI systems. By framing planning as an iterative and costly computational process, the study sheds light on the value of effective hierarchical planning structures seen in both human cognition and AI systems. Furthermore, clustering states based on partial plan similarity offers insights into option-like representations vital for hierarchical learning in AI.

Conclusion

The study presents a significant advancement in understanding human meta-planning and its parallels in artificial intelligence. By modeling planned information processing with a robust theoretical framework validated through experimental data, the research offers vital insights into why and how humans optimize their cognitive resources. The work underscores the need for AI systems to incorporate meta-planning akin to human cognitive processes, enhancing their efficiency and sophistication in dynamically changing environments.