Memory, Space, and Planning: Multiscale Predictive Representations (2401.09491v2)

Abstract: Memory is inherently entangled with prediction and planning. Flexible behavior in biological and artificial agents depends on the interplay of learning from the past and predicting the future in ever-changing environments. This chapter reviews computational, behavioral, and neural evidence suggesting these processes rely on learning the relational structure of experiences, known as cognitive maps, and draws two key takeaways. First, that these memory structures are organized as multiscale, compact predictive representations in hippocampal and prefrontal cortex, or PFC, hierarchies. Second, we argue that such predictive memory structures are crucial to the complementary functions of the hippocampus and PFC, both for enabling a recall of detailed and coherent past episodes as well as generalizing experiences at varying scales for efficient prediction and planning. These insights advance our understanding of memory and planning mechanisms in the brain and hold significant implications for advancing artificial intelligence systems.

• The paper establishes that cognitive maps in key brain regions encode multiscale predictive representations essential for strategic planning.
• It integrates computational frameworks like the successor representation with behavioral evidence to demonstrate memory's role in guiding decision-making.
• The study highlights how offline replay and multi-step prediction insights can inform AI models that mimic human navigation and memory functions.

Overview of Multiscale Predictive Representations

The Integral Role of Memory in Planning

In an effort to understand how biological and artificial systems utilize memory for prediction and planning, research explores the concept of cognitive maps—spatial mental models aiding in navigation and problem-solving. Findings suggest that cognitive maps manifest as multiscale, predictive representations within the hippocampus and prefrontal cortex. This arrangement allows agents to recall past experiences vividly and generalize across different situations for effective forecasting and strategizing.

Cognitive Maps and Predictive Learning

Cognitive maps enable agents to navigate and infer relationships within their environment, drawing on the analogy of spatial navigation. These cognitive constructs are not merely reflections of past interactions but are structured to predict future events. The evidence suggests that memories are encoded at varying scales within the brain, corresponding to the complexity and abstraction level of tasks at hand. This multifaceted representation is critical for adaptive behavior, allowing for choices that range from detailed to broadly conceptual based on the context.

Computational Perspectives on Memory Organization

The discussion transitions into how these memory structures, particularly cognitive maps, can be understood through computational frameworks like reinforcement learning (RL). One method, the successor representation (SR), is notable for capturing multi-step, predictive representations, unlike traditional RL models that learn associations step by step. SR emphasizes a predictive aspect, where each memory activation cues the potential of future states. This aligns with recent behavioral and neural studies, positing that such representations could underlie complex memory tasks and decision-making processes.

Memory in Action: Behavioral and Neural Corroboration

Neuroscience research corroborates this theory. For instance, studies show multiscale predictive representations in hippocampal and prefrontal hierarchies during tasks that mimic real-life conditions like navigating a virtual city. Such evidence highlights the brain's ability to use these predictive memory structures to influence goal-directed behavior. Additionally, investigations into offline replay during rest times tie into the brain's process of updating plans based on past experiences, further supporting the theory. The successor representation potentially unites spatial navigation with episodic recall, suggesting a universal principle for memory organization across biological and artificial cognizers.