CORE Planner: Contextual Memory RL

• CORE Planner is a reinforcement-learning framework that integrates contextual memory for robust navigation in unknown, partially observable environments.
• It employs a sparse visibility graph and Transformer network to efficiently encode spatial structure and historical motion, mitigating local optima like oscillatory deadlocks.
• Empirical results show significant gains in travel distance, inference speed, and sim-to-real transfer, outperforming traditional and learning-based navigation baselines.

Contextual-memory Oriented Reinforcement-learning (CORE) Planner is a reinforcement-learning-based navigation framework for unknown environments in which a robot must reach a predefined goal without any prior map while continuously updating its belief about free space, obstacles, and unknown regions. The method combines a sparse visibility graph for structured environment representation, a Transformer network for holistic environmental understanding, a graph sparsification method for large-scale scenes, and a contextual memory mechanism that encodes historical motion to alleviate local optima and improve navigation efficiency. In the reported experiments, CORE Planner is trained solely on image-based environments and transfers to real robots in a zero-shot manner, with no fine-tuning (Kong et al., 28 Jun 2026).

1. Problem setting and design rationale

CORE Planner is formulated for autonomous navigation in unknown environments under partial observability. At each step, the robot has only a local and incomplete belief of the world, and must infer useful next-waypoint decisions from evolving sensor observations. The paper emphasizes a characteristic failure mode of such settings: a waypoint that appears locally promising may later become invalid or lead into a dead end, producing back-and-forth motion or oscillation. This failure is described as Navigation Deadlock Oscillation (NDO), especially in room-to-room navigation, where the robot may repeatedly alternate between apparently favorable waypoints (Kong et al., 28 Jun 2026).

The method is motivated by limitations in two established families of navigation systems. Traditional planners, including Dijkstra, A*, D*, LPA*, D* Lite, RRT variants, FAR Planner, E-Planner, and FPS, rely on hand-crafted rules and heuristic costs; dense occupancy-grid representations can also become computationally heavy in large unknown environments. Prior learning-based planners, including NavRL, YOPO, CTSAC, CADRL, and GDAE-style hybrid planners, often use fixed-size CNN or RGB-D inputs, have short memory or shallow sensor history, and are frequently local planners rather than long-horizon planners. CORE Planner is presented as combining the sparse, structured geometry of visibility graphs with learned decision-making and a contextual memory mechanism that records historical motion, thereby addressing both local optima and scalability (Kong et al., 28 Jun 2026).

A central conceptual move is to reject purely greedy waypoint selection. The classical next-waypoint objective is written as

$$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(t-1), B(t-1)\big),$$

where $\mathcal{P}(t)$ is the set of admissible next waypoints or local plans. CORE extends this to a history-aware objective,

$$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$

but does not literally preserve the full trajectory in raw form; instead, it compresses history into node-wise contextual memory (Kong et al., 28 Jun 2026).

2. Formal RL formulation

The environment is modeled as a $2$D occupancy grid map $E$, partitioned into free space $E_f$ and obstacle space $E_o$. The robot maintains a belief over believed obstacle space $B_o$, unknown space $B_u$, and believed free space $B_f$, and updates this belief through sensor measurements $\mathcal{P}(t)$0 according to

$\mathcal{P}(t)$1

This provides the continuously updated partial world model on which planning operates (Kong et al., 28 Jun 2026).

CORE Planner is cast as an MDP,

$\mathcal{P}(t)$2

with policy objective

$\mathcal{P}(t)$3

After constructing a visibility graph, the action is to choose one of the neighboring visible nodes as the next waypoint. Because the candidate set is variable-sized, the action selector is implemented as a Pointer Network (Kong et al., 28 Jun 2026).

The reward is a weighted sum of five terms,

$\mathcal{P}(t)$4

The goal-distance term encourages progress toward the goal,

$\mathcal{P}(t)$5

The frontier term rewards newly observed frontier points,

$\mathcal{P}(t)$6

The position-difference term penalizes stagnation,

$\mathcal{P}(t)$7

The stagnation penalty is

$\mathcal{P}(t)$8

and the finish reward is

$\mathcal{P}(t)$9

Taken together, these terms reward goal progress, exploration, movement continuity, escape from deadlock, and task completion (Kong et al., 28 Jun 2026).

3. Visibility-graph state representation and contextual memory

CORE Planner represents the navigation state at time $$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$0 as a graph

$$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$1

where $$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$2 are nodes and $$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$3 are edges between collision-free visible nodes. Each node $$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$4 carries four components: spatial coordinates $$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$5, utility $$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$6, distance to goal $$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$7, and trajectory indicator $$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$8. The corresponding node feature is

$$p^*(t)=\arg\min_{p\in\mathcal{P}(t)}\; \mathrm{Cost}\!\big(p;\; S(0),\ldots,S(t-1), B(t-1)\big),$$9

The use of a visibility graph rather than a dense occupancy grid is justified as a topological abstraction that retains essential geometry and connectivity while removing redundant spatial cells, thereby reducing computational overhead in large-scale exploration (Kong et al., 28 Jun 2026).

The paper adds a further graph sparsification stage. Nodes within a non-sparsified region around the robot are kept unchanged, while farther nodes are clustered using a distance threshold and visibility-connectivity constraints, then replaced by centroid representatives. For a cluster $2$0, the centroid is

$2$1

The resulting complexity is described as

$2$2

in contrast to the non-sparsified

$2$3

The paper notes that $2$4 is empirically small, often $2$5 in large scenes (Kong et al., 28 Jun 2026).

The contextual memory mechanism is encoded through the trajectory indicator

$2$6

which counts how many times the robot has visited or come near node $2$7. This is explicitly more informative than a binary visited or unvisited flag. Because $2$8 is embedded directly into $2$9, the policy can identify repeatedly chosen dead-end regions, suppress cyclic behavior, and maintain long-term consistency in path reasoning. The paper argues that this augmentation preserves Markovity by defining an augmented state space

$E$0

with transition

$E$1

since the memory update is deterministic given actions (Kong et al., 28 Jun 2026).

4. Transformer architecture and SAC training

The network backbone is a Transformer encoder-decoder operating on graph-structured inputs. Node features are stacked as

$E$2

then projected into latent space through

$E$3

where $E$4 and $E$5. A graph-based mask $E$6 constrains encoder attention so that nodes attend according to graph connectivity rather than globally. The encoder output, denoted $E$7, is described as environment-aware features combining spatial structure, node utilities, goal distance, and visitation memory. In the decoder, the current robot node feature $E$8 serves as the query, while $E$9 serves as keys and values. The pointer attention layer converts the decoded representation and neighboring node embeddings into attention weights $E_f$0, which are directly interpreted as the action distribution over next waypoints (Kong et al., 28 Jun 2026).

Training uses Soft Actor-Critic (SAC). The paper gives the soft Q-function

$E_f$1

the soft value function

$E_f$2

the soft Bellman operator

$E_f$3

and the entropy-regularized objective

$E_f$4

The paper states that convergence is preserved because node features are bounded, $E_f$5, and the augmented state remains Markovian (Kong et al., 28 Jun 2026).

The training environment uses the environment generator from Chen et al., with map size $E_f$6 grid, physical size $E_f$7, and sensor range $E_f$8. Training includes simple maps and complex maps; testing uses only complex unseen configurations. The reported optimization details are maximum episode length $E_f$9, Adam, learning rate $E_o$0, replay buffer size $E_o$1, and batch size $E_o$2, on an Intel Xeon E5-2660 v4 CPU and $E_o$3 NVIDIA RTX 2080 Ti GPUs. Training is reported to converge in about five hours in an image-based simulated environment, without Gazebo or Isaac Sim (Kong et al., 28 Jun 2026).

5. Empirical performance, ablations, and sim-to-real transfer

The evaluation covers image-based unseen environments, Gazebo simulation, and real-world deployment, using success rate $E_o$4, travel distance $E_o$5, inference time $E_o$6, planning time $E_o$7, and human interventions $E_o$8 as metrics. Across these settings, the paper reports that CORE Planner consistently outperforms the traditional FAR Planner and all learning-based baselines, with larger gains in more complex environments (Kong et al., 28 Jun 2026).

Setting	Baseline(s)	Reported CORE outcome
Image-based unseen environments	CADRL: success $E_o$9, distance $B_o$0, time $B_o$1	success $B_o$2, distance $B_o$3, time $B_o$4; travel distance reduced by 20.8%
Gazebo simulation	FAR Planner, CTSAC, NavRL, YOPO	up to 13% reduction in travel distance over FAR Planner; simple indoor: 18.3% vs CTSAC, 23.3% vs NavRL, 14.7% vs YOPO; complex indoor: 37.1% vs YOPO; forest: 48% vs NavRL and 30.1% vs YOPO
Real-world Scout Mini	FAR: $B_o$5, $B_o$6, $B_o$7	$B_o$8, $B_o$9, $B_u$0
Real-world LYNX M20	No quantitative FAR comparison given	completed all trials without intervention

The image-based result is notable because the baseline CADRL also performs strongly, with $B_u$1 success, yet CORE reaches $B_u$2 and shortens travel distance from $B_u$3 to $B_u$4. In Gazebo, the paper highlights that CTSAC and NavRL fail in the complex indoor setting, and CTSAC fails in the forest setting, whereas CORE remains operational. Inference time for CORE is reported as around $B_u$5 ms in the simple indoor environment and around $B_u$6 ms in the forest environment, while FAR Planner’s inference time grows from $B_u$7 in Simple to $B_u$8 in Forest, which the authors attribute to CORE’s sparse topological representation (Kong et al., 28 Jun 2026).

The ablation study isolates the two main modules. CORE w/o Sparse retains contextual memory but removes graph sparsification; CORE w/o Mem retains sparsification but removes contextual memory. In the simple scenario, the variants behave similarly because sparsification is rarely activated. In larger environments, however, removing memory causes substantial degradation. In Indoor, CORE w/o Mem produces $B_u$9 m travel distance with $B_f$0 interventions, versus CORE at $B_f$1 m and $B_f$2 interventions. In Forest, CORE w/o Mem yields $B_f$3 m versus CORE at $B_f$4 m. The paper interprets this as evidence that sparsification primarily improves efficiency, whereas contextual memory is responsible for robustness against oscillatory motion and NDO (Kong et al., 28 Jun 2026).

The real-world experiments use two different platforms: a LYNX M20 quadruped with a ZED2 RGB-D camera and NVIDIA Jetson AGX Orin, and a Scout Mini UGV with a Livox Mid-360 LiDAR and NVIDIA Jetson Orin NX. Tasks include multi-room local-optimum escape and dynamic obstacle avoidance. The reported zero-intervention performance on both platforms is presented as evidence of cross-sensor, cross-robot, and cross-hardware generalization under zero-shot sim-to-real transfer (Kong et al., 28 Jun 2026).

6. Position within contextual-memory RL research and open issues

Within the recent literature, CORE Planner belongs to a broader class of architectures that treat planning quality and memory organization as central design variables rather than auxiliary modules. In long-horizon digital-task automation, a planner-actor-memory decomposition has been used to show that scaling the planner yields the largest marginal gain, while execution and memory management can remain smaller; the same work reports that planner-only GRPO training with frozen actor and memory manager is the most stable RL strategy (Wu et al., 4 May 2026). In memory-augmented LLM systems, Mem-$B_f$5 formulates memory construction itself as a sequential decision problem over memory_insert, memory_update, and memory_delete, with GRPO optimizing a write policy over core, episodic, and semantic memory (Wang et al., 30 Sep 2025). CoDA likewise separates a short strategic Planner context from a temporary Executor context to mitigate “Context Explosion,” using PECO for joint RL optimization of both roles (Liu et al., 14 Dec 2025). These results suggest a more general research tendency: contextual memory is increasingly being treated as a mechanism for preserving strategic state while limiting the computational and statistical costs of long-horizon decision making.

The acronym CORE is also used in a distinct line of work on active causal discovery, where it denotes Causal DiscOvery with REinforcement learning rather than navigation. That method formulates causal discovery as a POMDP and uses history-based context together with dual DQN controllers for intervention and structural actions (Sauter et al., 2024). The naming overlap is terminological rather than methodological; the robot-navigation CORE Planner is specifically the Contextual-memory Oriented Reinforcement-learning framework described above.

The navigation paper identifies a concrete limitation: in real-world scenarios, visibility graph extraction may become less reliable in complex environments with irregular geometry, which can make structural nodes less informative and cause failures. The authors also indicate future work on integrating semantic information into the navigation framework (Kong et al., 28 Jun 2026). A plausible implication is that future versions of CORE Planner may move toward richer memory representations that combine geometric topology, trajectory history, and semantic cues, thereby narrowing the gap between robot navigation and the planner-centric contextual-memory systems now appearing in language-agent research.