HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control

Abstract: Current Vision-Language-Action (VLA) models excel at robotic manipulation but often struggle with non-Markovian tasks requiring long-term memory and reasoning due to their reliance on immediate observations. Existing solutions face a ''frequency-competence paradox,'' where stronger reasoning models are too slow for real-time control, while faster models lack sufficient reasoning capabilities. To resolve this architectural misalignment, we propose HiMe, a Hierarchical Embodied Memory framework that decouples embodied intelligence into a high-frequency Executor for execution, a Sentry for working memory, and a Planner for long-term strategy. We also introduce a dynamic knowledge system based on cross-modal semantic schemas and active management mechanisms, allowing robots to maintain memory plasticity through ''Add, Update, and Delete'' operations. This hierarchical design effectively balances the conflict between real-time execution and slow thinking planning, significantly improving success rates in long-horizon tasks. Experiments demonstrate that this approach not only outperforms flat memory baselines but also exhibits the novel ability to self-correct its internal knowledge based on human preferences.

• The paper introduces a hierarchical memory system that decouples real-time control and high-level reasoning using Executor, Sentry, and Planner.
• The approach achieves a 90% success rate on complex tasks while reducing planner API calls by up to 3x through active memory management.
• Cross-modal memory and explicit Add/Update/Delete operations ensure robust, consistent long-horizon performance in non-Markovian environments.

HiMe: Hierarchical Embodied Memory for Long-Horizon Vision-Language-Action Control

Motivation and Problem Formulation

Vision-Language-Action (VLA) models have demonstrated strong manipulation capabilities in robotic domains with large-scale pretraining, but are fundamentally limited in long-horizon and non-Markovian settings where global context, memory retention, and reasoning are required. The main bottleneck is the frequency-competence paradox: high-capacity vision-language planners are unable to operate at the real-time frequencies required for physical control, while low-latency controllers lack sufficient reasoning and memory. Prior attempts either augment VLA policies with memory banks of limited scale or periodically retrain high-level planners, but both approaches suffer from noisy context, catastrophic forgetting, and inefficiency.

The HiMe framework addresses this architectural misalignment by explicitly decoupling embodied intelligence into three temporally-granular functional layers: a high-frequency Executor for fine-grained actuation, a Sentry for transient working memory and progress monitoring, and a Planner for episodic memory and long-horizon reasoning. The system is augmented with cross-modal semantic schemas and active management protocols to ensure memory plasticity and alignment in highly dynamic, user-driven tasks.

Figure 1: Overview of HiMe and the motivating example for task-relevant multimodal memory maintenance across long-horizon subtasks.

Architecture and Hierarchical Memory Design

The core architectural innovation in HiMe is the hierarchical decomposition of control and memory:

• Executor ($\pi_e$): Stateless, high-frequency VLA policy, conditioned on the current subgoal. Responsible for real-time sensory-motor coordination with minimal context; operates efficiently by circumventing episodic memory queries.
• Sentry ($\pi_s$): Lightweight vision-language module operating on a sliding window of recent frames. Periodically checks subtask completion and signals task transitions. By deferring high-level planning until relevant transitions, the Sentry mitigates unnecessary replanning and reduces susceptibility to transient noise.
• Planner ($\pi_p$): Heavyweight multimodal module, invoked only upon Sentry’s trigger. Accesses rich episodic memory, which is structured as contextual key-value stores (visual + textual) and procedural plans (ordered subgoal sequences), and performs complex retrieval, consolidation, and memory refinement via explicit Add, Update, and Delete operations.

  Figure 2: HiMe architecture. The closed-loop embodies separation of execution and high-level reasoning, with the Sentry monitoring progress and the Planner updating memory.

This hierarchy enables amortized reasoning, ensures that critical strategic context is maintained, and provides robustness against the limitations of fixed-context VLM policies. The memory backend utilizes semantic embeddings and vectorized retrieval, facilitating data-efficient memory query and manipulation.

Experimental Evaluation and Ablations

Three tabletop manipulation tasks were designed: Object Search (active inspection, belief updating), Counting (persistent semantic and progress tracking), and Rearrangement (long-term spatial recall). The evaluation protocol precisely measures task progress, planner invocation count, and memory retrieval efficacy over 20 trials per task and method.

Figure 3: Task suite used for evaluation, covering dynamic inspection, long-horizon counting, and preference-driven rearrangement.

HiMe achieves a 90% average success rate, outperforming all baselines, including transient memory, flat memory, and Sentry-only variants. Crucially, its performance closely approaches the human oracle in high-level subtask specification, validating the effectiveness of structured, actively-managed memory. The Sentry mechanism enhances task consistency by reducing erratic task switching and enables high data quality for memory consolidation.

Figure 4: HiMe outperforms transient, Sentry, and flat memory baselines in long-horizon manipulation, bridging the gap to the human oracle.

Modality ablations further demonstrate that cross-modal memory (image + text) yields superior performance: image-only memory performs better in spatial tasks, text-only in semantic reasoning, but neither achieves the level of interleaved memory.

Figure 5: Ablation confirms necessity of cross-modal memory: interleaving image/text achieves generalization across spatial and semantic tasks.

Memory management ablations confirm the necessity of active protocols. FIFO and passive accumulation are consistently inferior; even unconstrained history fails to match actively curated memory, with the latter maintaining concise, consistent context and eliminating obsolete entries.

Figure 6: Memory management ablation: explicit Add/Update/Delete enables higher task progress across all tasks by avoiding obsolete or redundant states.

Real-World Deployment and Scalability

HiMe was instantiated on a WidowX-250 robotic platform with dual-camera input and task objects. The Executor was fine-tuned from $\pi_{0.5}$ (trained on DROID), the Planner uses GPT-4o or Qwen3-VL-30B, and the Sentry uses Qwen3-VL-8B for online monitoring, ensuring both grounding and reasoning capacity. Real-robot experiments validate HiMe's robustness, real-time execution compatibility, and extendability to open-source planners.

Figure 7: Real-world experimental setup with robot arm, cameras, workspace, and storage containers.

Additional experiments confirm scalability: task success rates decrease with increasing horizon, highlighting the challenge of memory maintenance; Planner latency is decoupled from high-frequency control, and local deployment is feasible.

Figure 8: Qwen3-VL-30B open-source Planner matches main results: HiMe remains highest-performing across modality and memory ablations.

Analysis and Key Results

• Efficiency: HiMe reduces Planner API calls by up to 3x compared to flat memory; memory hit rates remain above 94%, minimizing redundant exploration and latency.
• Consistency: Active memory management prevents catastrophic forgetting, ensures reliable context retrieval, and improves long-horizon reasoning.
• Plasticity: Explicit Add/Update/Delete operations allow self-correction under evolving user preferences and environment changes.
• Modality Alignment: Cross-modal semantic schemata enable both spatial grounding and complex logic integration.
• Empirical Upper Bound: HiMe nearly matches human-level high-level planning, given fixed Executor capacity.

Implications and Future Directions

HiMe's approach to hierarchical embodied memory allows practitioners to resolve the real-time reasoning bottleneck without sacrificing context breadth or computational efficiency. Architecturally, it enables robotic agents to operate in persistently non-Markovian domains, adapt to user-driven environmental changes, and maintain consistent internal representations across extensive action horizons.

Practical implications include:

• Modular deployment of memory-enabled agents in service and assistive robotics.
• Plug-and-play compatibility with open-source planners and local serving.
• Extension to larger, more dexterous manipulation domains as demonstrated by current baselines.

Theoretically, HiMe represents a shift from passive, transient memory models to dynamically-managed, multimodal knowledge systems. This offers a pathway to autonomous, adaptation-capable agents that can maintain high-level semantic consistency and learn from evolving interaction histories.

Further research is warranted in:

• Integrating the framework with LLM-driven multi-agent orchestration.
• Scaling to diverse, real-world environments and more complex physical primitives.
• Benchmarking against standard simulation tasks to systematically characterize long-horizon capabilities.

Conclusion

HiMe establishes a principled solution to the frequency-competence paradox in long-horizon VLA control by formalizing hierarchical memory separation and active management. Empirical results validate substantial gains in task completion rate, efficiency, and robustness. The cross-modal memory and active management components are indispensable for maintaining consistent, concise, and adaptive internal knowledge in non-Markovian settings. This architecture will underpin further advances in memory-centric embodied agents, both in research and practical robotic deployments.

---

Figure 1: Overview of HiMe: hierarchical separation of fast execution and memory-driven reasoning for long-horizon subtasks.

Figure 2: HiMe architecture: closed-loop system with Executor, Sentry trigger, and Planner memory consolidation.

Figure 3: Task suite for evaluation including inspection, counting, and rearrangement.

Figure 4: HiMe main results: achieves 90% task progress and matches human high-level planning.

Figure 5: Ablation of modality: cross-modal memory outperforms text- and image-only variants.

Figure 6: Ablation of management: active memory yields higher progress compared to FIFO and passive accumulation.

Figure 7: Real-world experimental deployment with robotic manipulators and workspace.

Figure 8: Additional experiments using Qwen3-VL-30B: consistent superiority of HiMe across baselines.