CORE Planner: Contextual-memory Oriented Reinforcement-learning in Unknown Environments for Robot Navigation

Abstract: Autonomous navigation in unknown environments requires a robot to efficiently reach a predefined goal while exploring without prior maps. Although progress has been made in this area, most existing works still rely on traditional planning methods with hand-crafted rules, while learning-based methods often suffer from limited environmental memory and challenges in simulation-to-real (sim-to-real) transfer. To overcome these limitations, we propose a Contextual-memory Oriented Reinforcement-learning (CORE) planner for robot navigation in unknown environments. The proposed CORE planner effectively combines the core advantages of traditional and learning-based methods. Specifically, our method uses a sparse visibility graph for structured environment representation, reducing the computational overhead of dense grid maps, and employs a Transformer network to achieve a holistic environmental understanding, thereby significantly improving navigation efficiency. Moreover, we introduce a visibility graph-based graph sparsification method and a contextual memory mechanism, which alleviates local optima and enhances computational performance in large-scale scenes. Finally, our approach achieves zero-shot sim-to-real transfer after training solely on image-based environments, requiring no fine-tuning. Experimental results show that CORE Planner consistently outperforms state-of-the-art methods, including the traditional FAR Planner and all learning-based baselines, across representative environments, reducing travel distance by 13\% over traditional FAR Planner and by up to 48\% relative to learning-based baselines, with larger gains observed in more complex environments. In real-world scenarios, CORE successfully navigates without human intervention, showcasing zero-shot sim-to-real transfer. Code is available at https://github.com/BBD00/core_planner.

• The paper presents CORE planner, which integrates sparse visibility graphs with a contextual memory mechanism to enable zero-shot sim-to-real transfer in unknown environments.
• It employs a Transformer encoder-decoder with a pointer network to reduce computational overhead and improve path planning, achieving up to 48% shorter paths compared to baselines.
• Experimental evaluations in simulation and real-world deployments demonstrate robust, deadlock-free navigation with reduced travel distances and zero human interventions.

CORE Planner: Contextual Memory-Oriented Reinforcement Learning for Efficient Robot Navigation in Unknown Environments

Introduction

Addressing autonomous navigation in unknown environments, this paper introduces CORE planner, a reinforcement learning-based framework that integrates sparse visibility graph representations with a contextual memory mechanism. The method is explicitly designed to overcome major limitations of both conventional hand-crafted planning routines and learning-based models: the former’s reliance on predefined rules and susceptibility to local minima, and the latter’s poor environmental memory and limited sim-to-real transfer capability. CORE planner augments structured geometric reasoning with Transformer-based context integration, allowing zero-shot transfer from simulation to real-world robotic platforms while maintaining computational efficiency even in large-scale, partially observable environments (2606.29222).

Figure 1: The unknown-environment navigation problem, with the robot incrementally exploring uncharted space using local onboard sensing; context-aware choices facilitate escape from deadlocks and efficient progress.

Methodology

Sparse Visibility Graph Representation and Dynamic Graph Sparsification

CORE planner exploits a visibility-graph-based abstraction in lieu of traditional dense grid maps. This graph only retains key geometric features (robot/goal/frontiers, visible obstacles) and essential connectivity, substantially reducing representation and processing overhead. To ensure scalability, a dynamic graph sparsification algorithm further clusters distant, interconnected nodes beyond an immediate local ring, reducing the quadratic complexity of attention-based processing and ensuring real-time operation, especially in large environments.

Contextual Memory Embedding

Each node in the visibility graph is enriched with an explicit trajectory indicator—the count of robot visits—encoding contextual action history and enabling the network to distinguish between unexplored and repeatedly traversed regions. Unlike short fixed-history buffers, this mechanism supports unbounded contextual memory and is compactly stored at the node level, preserving Markov properties for RL convergence and drastically reducing oscillatory “deadlock” behaviors in long-horizon partial observability scenarios.

Transformer Encoder-Decoder and Pointer Network

The visibility graph augmented with contextual memory is processed by a Transformer encoder-decoder. The encoder operates over graph-local attention masks, learning global and historical dependencies. The decoder attends to the current agent node and neighbors; a pointer network outputs a distribution over feasible neighboring waypoints (actions). Soft Actor-Critic loss with a compound reward (goal distance, frontier expansion, movement, stagnation, task completion) underpins reinforcement learning.

Figure 3: The CORE planner pipeline, combining incremental sparse visibility graph construction, contextual memory, Transformer encoding, and an attention-based pointer network for waypoint selection.

Theoretical Guarantees and RL Formulation

CORE planner’s underpinning RL process is shown to inherit the convergence guarantees of entropy-regularized policy gradient methods (SAC), due to boundedness of graph features and preservation of the MDP properties even after augmentation with contextual memory. The policy optimizes the expected entropy-regularized cumulative reward in the high-dimensional, partially observable settings inherent to unknown environments.

Experimental Evaluation

Simulated Environments and Baseline Comparison

CORE planner was trained solely in image-based, randomly generated maps combining simple and complex geometric elements. Zero-shot evaluation in 100 complex, unseen environments demonstrates a 100% navigation success rate, with a notable 20.8% reduction in travel distance compared to the state-of-the-art CADRL learning-based baseline.

Figure 2: Training on both simple and complex environments; evaluation is exclusively on complex, unseen maps.

Figure 4: Navigation trajectory comparison (a) CORE planner vs. (b) CADRL, indicating shorter paths and absence of oscillations for CORE.

Gazebo Simulation and Comprehensive Ablations

Comparative analysis in Gazebo simulation (simple/complex indoor, outdoor forest) shows CORE planner outperforms FAR (visibility graph-based planner) and all modern RL-based baselines. Notably, CORE achieves a 13% reduction in travel distance over FAR and up to 48% against learning-based baselines in the most complex (forest) scenarios. CORE consistently requires zero human interventions, whereas traditional and RL baselines display failures due to oscillatory deadlocks or failure to generalize to larger or non-static environments.

Figure 5: Sequential waypoint navigation in (a) simple, (b) complex indoor, and (c) complex forest environments, showing consistently shortest core-generated paths.

Ablation confirms that disabling contextual memory reintroduces oscillatory behaviors and increases path length by up to 52% in indoor settings. Graph sparsification is critical for maintaining low inference latency in large-scale environments.

Real-world Robotic Deployment and Sim-to-Real Transfer

CORE was deployed on two physically distinct platforms (wheeled Scout Mini with LiDAR, quadruped LYNX M20 with RGB-D), requiring no real-world retraining. In a multi-room environment with local optimal traps, CORE completed navigation with 0 human interventions and 17.5% shorter path than FAR, which failed repeatedly due to deadlock entrapment.

Figure 6: Experimental platforms—Scout Mini (LiDAR, wheeled) and LYNX M20 (RGB-D, quadruped), validating cross-modality transfer.

Figure 7: Real-world multi-room navigation, with (a) CORE planner escaping local optimum autonomously; (b) FAR Planner becomes trapped and requires human intervention.

In dynamic obstacle navigation with the LYNX M20 (RGB-D only), CORE maintained stable progress and effective avoidance behaviors, evidencing transferability across modalities, morphologies, and embedded hardware.

Figure 8: LYNX M20 navigating static and dynamic obstacles in a forest, demonstrating real-time responsiveness and transfer from grid-based simulation to RGB-D-based real-world execution.

Implications and Outlook

The demonstrated zero-shot sim-to-real transfer without fine-tuning, robust avoidance of navigation deadlocks, and consistent computational efficiency even in large environments represent decisive advances in RL-based navigation. By abstracting away map-scale input dimensionality in favor of topological, memory-rich representations and Transformer-based reasoning, CORE planner circumvents previous limitations of scalability, overfitting, and local minima that characterize both hand-crafted and naïve RL approaches.

The method’s performance underscores the benefit of non-parametric contextual memory and sparse graphical abstractions for embodied decision-making under partial observability. Practically, CORE is immediately applicable to real-world mobile robots operating without prior maps in unstructured and variable environments using on-board compute.

Theoretically, integrating semantic scene understanding into the existing geometric-contextual framework is a natural future direction, raising the prospect of context-aware, high-level task specification with tight low-level control in unknown settings. Further, the modularity of the architectural backbone means that advances in graph-based neural reasoning, RL objectives, or state encoding can be incorporated into the pipeline.

Conclusion

CORE planner establishes a new baseline for unknown-environment navigation, realizing joint geometric and contextual reasoning via reinforcement learning on sparse visibility graphs and explicit trajectory memory. The empirical results point to strong gains in navigation efficiency, reliability, and sim-to-real robustness over both traditional and learning-based benchmarks, supporting the broader adoption of context-aware, topological RL frameworks for scalable real-world robotic navigation (2606.29222).