OVAL: Open-Vocabulary Augmented Memory Model for Lifelong Object Goal Navigation

Abstract: Object Goal Navigation (ObjectNav) refers to an agent navigating to an object in an unseen environment, which is an ability often required in the accomplishment of complex tasks. While existing methods demonstrate proficiency in isolated single object navigation, their limitations emerge in the restricted applicability of lifelong memory representations, which ultimately hinders effective navigation toward continual targets over extended periods. To address this problem, we propose OVAL, a novel lifelong open-vocabulary memory framework, which enables efficient and precise execution of long-term navigation in semantically open tasks. Within this framework, we introduce memory descriptors to facilitate structured management of the memory model. Additionally, we propose a novel probability-based exploration strategy, utilizing a multi-value frontier scoring to enhance lifelong exploration efficiency. Extensive experiments demonstrate the efficiency and robustness of the proposed system.

• The paper introduces a modular framework that leverages open-vocabulary memory and probability-based exploration to enable lifelong object goal navigation.
• It employs a rich memory model integrating visual features, spatial descriptors, and synonym-aware querying for efficient instance management.
• Empirical results show significant enhancements in success rate and SPL, validating the approach for continuous and scalable semantic navigation.

OVAL: Open-Vocabulary Augmented Memory Model for Lifelong Object Goal Navigation

Introduction

Object Goal Navigation (ObjectNav) in embodied agents demands semantic understanding and efficient spatial reasoning to locate user-specified objects in unfamiliar environments. Traditional approaches, while effective for isolated single-step tasks, fundamentally lack robust lifelong memory models, rendering them ill-suited for complex, sequential, open-vocabulary scenarios. The paper "OVAL: Open-Vocabulary Augmented Memory Model for Lifelong Object Goal Navigation" (2604.12872) introduces a framework that jointly addresses the requirements of open-vocabulary semantic generalization and persistent, structured memory—enabling robots to perform continual goal-directed navigation in large-scale, dynamically instructed scenarios.

System Architecture

The OVAL system adopts a modular architecture integrating three primary functional components: a probability-driven frontier exploration module, an open-semantic memory model with advanced instance management, and a navigation module employing robust semantic querying and verification strategies.

Figure 1: The OVAL pipeline integrates frontier-based exploration, open-semantic memory modeling, and synonym-aware navigation and verification.

The exploration module builds a topological grid with depth and pose, dynamically scores frontiers based on a multi-factor probability map, and guides agents toward regions promising efficient coverage and goal discovery. Simultaneously, the open-semantic memory subsystem abstracts and manages open-vocabulary instance descriptors, supporting robust feature-based object representation and high-fidelity recall. The navigation engine employs substring and synonym expansion (via KMP queries and LLM-generated paraphrase clusters) to bridge lexical granularity mismatches, supports greedy waypoint planning, and verifies success with panoramic imaging and weighted LLM-based semantic validation.

Frontier Exploration with Probability-Based Scoring

Exploration efficiency and coverage are maximized using a probability-based frontier selection strategy constructed from three orthogonal information sources:

• Distance Component ($o_d(F)$): Prefers proximal frontiers to minimize trajectory cost and improve SPL.
• Semantic Component ($o_s(F)$): Exploits map-level context by leveraging semantic priors (e.g., presence of co-occurring objects) to direct search toward semantically relevant subregions.
• Footprint Component ($o_f(F)$): Penalizes frequently revisited locations to enforce coverage and prevent oscillation.

A Gaussian-weighted sum (with adaptive amplitude and spatial decay) aligns these factors, allowing the agent to adaptively balance local efficiency and global exploration requirements.

Open-Semantic Memory Model and Instance Management

Unlike prior systems reliant on raw text labels or rigid category dictionaries, OVAL employs a rich memory representation integrating:

• Object labels $S_i$ subject to filtered autolabeling and synonym grouping
• Visual feature buffers ($\mathcal{I}_i$)
• Spatial descriptors ($X_i$)
• HSV-based scene histograms ($H_i$)
• Confidence scores ($C_i$) quantifying observation geometry and detection quality

Memory model management handles instance discrimination via a staged similarity assessment, first comparing scene and positional descriptors and then, in ambiguous cases, invoking local feature correspondence (using SuperGlue) for robust instance clustering.

Figure 2: Hierarchical memory model management ensures new instances are accurately merged or instantiated, maintaining retrieval fidelity and manageable storage complexity.

This structured memory, alongside synonym-aware retrieval, allows for efficient and accurate recall—even in highly dynamic, open-vocabulary environments.

Navigation: Goal Query and Stop Verification

To resolve semantic mismatches between user instructions and detected object phrases, OVAL integrates KMP-based substring search and LLM-powered synonym expansion, facilitating robust cross-granularity querying. Upon candidate retrieval, navigation proceeds via waypoint planning, with success verified using a hybrid of panoramic LLM-based presence estimation and the memory model's confidence signal.

Empirical Performance

Experiments performed in the Habitat simulation suite (covering HM3D and MP3D datasets) evaluate both lifelong ObjectNav and traditional single-episode ObjectNav. OVAL is compared to SOTA training-free, open-vocabulary, and memory-augmented baselines (e.g., VLFM [vlm1], GOAT [goat]). Metrics include SR and SPL.

Figure 3: Lifelong ObjectNav SR and SPL benchmarked against GOAT on HM3D, across increasing target list lengths.

Key quantitative findings:

• OVAL achieves an SR of 68.1% and SPL of 33.8% on HM3D lifelong ObjectNav, substantially outperforming prior methods (VLFM SR 53.7%, GOAT SR 59.2%).
• Robust operation despite open-vocabulary queries: GOAT fails on MP3D due to category set limitations, whereas OVAL sustains competitive results (SR 44.1%).
• Object memory and probability-based exploration contribute additive gains; ablations demonstrate SR/SPL degradation when these components are disabled.

  Figure 4: Ablation on number of lifelong targets: model performance stabilizes and improves as task complexity (sequential targets) increases, demonstrating scalable long-term recall.

The empirical analysis further highlights that the footprint penalization is most essential for reducing redundant exploration, while semantic cues promote task-specific efficiency.

Practical and Theoretical Implications

OVAL's explicit memory management enables lifelong open-vocabulary navigation by supporting:

• Continuous, non-destructive semantic accumulation across sequential tasks
• Memory-efficient representation and fast query even under high instance multiplicity
• Advanced generalization to unseen labels, reducing reliance on category pre-definition

Practically, this facilitates robust deployment in real-world service robotics (e.g., hotels, homes), where task instructions follow unpredictable, evolving open-vocabulary distributions. Theoretically, the framework demonstrates the importance of modular architecture, explicit memory, and multi-modal semantic reasoning for long-horizon embodied autonomy.

Future Directions

Current limitations include absence of multimodal input fusion and limited robustness to rapid environmental dynamics; integrating audio, textual, or tactile streams, as well as designing temporal abstraction for dynamic memory update, constitute promising directions. Further, improvements in memory compression and error-correction could facilitate even larger-scale lifelong deployments.

Conclusion

OVAL presents a technically rigorous, explicitly managed, open-vocabulary memory and navigation system for lifelong ObjectNav, setting new standards for SR and SPL on challenging benchmarks. The combination of probability-driven exploration, robust multi-descriptor memory modeling, and efficient synonym-aware querying constitutes a significant contribution to the state of persistent embodied navigation, with implications for scalable, adaptive deployment in open-world service scenarios.