AstraNav-Memory: Contexts Compression for Long Memory (2512.21627v1)

Abstract: Lifelong embodied navigation requires agents to accumulate, retain, and exploit spatial-semantic experience across tasks, enabling efficient exploration in novel environments and rapid goal reaching in familiar ones. While object-centric memory is interpretable, it depends on detection and reconstruction pipelines that limit robustness and scalability. We propose an image-centric memory framework that achieves long-term implicit memory via an efficient visual context compression module end-to-end coupled with a Qwen2.5-VL-based navigation policy. Built atop a ViT backbone with frozen DINOv3 features and lightweight PixelUnshuffle+Conv blocks, our visual tokenizer supports configurable compression rates; for example, under a representative 16$\times$ compression setting, each image is encoded with about 30 tokens, expanding the effective context capacity from tens to hundreds of images. Experimental results on GOAT-Bench and HM3D-OVON show that our method achieves state-of-the-art navigation performance, improving exploration in unfamiliar environments and shortening paths in familiar ones. Ablation studies further reveal that moderate compression provides the best balance between efficiency and accuracy. These findings position compressed image-centric memory as a practical and scalable interface for lifelong embodied agents, enabling them to reason over long visual histories and navigate with human-like efficiency.

• The paper introduces a novel image-centric memory system that compresses hundreds of visual observations to support long-horizon navigation.
• It employs a frozen DINOv3 ViT backbone with structured PixelUnshuffle and lightweight convolutions for efficient token compression.
• Experimental results on GOAT-Bench and HM3D-OVON demonstrate significant gains in success rate and path efficiency over traditional methods.

AstraNav-Memory: Contexts Compression for Long Memory

Introduction and Motivation

AstraNav-Memory addresses the problem of scaling lifelong embodied navigation in partially observable indoor environments, where agents must accumulate and exploit extensive visual experience spanning multiple tasks and instructions. Traditional object-centric memory frameworks, while interpretable, depend on explicit detection and semantic reconstruction pipelines that introduce brittleness and limit scalability across diverse domains. AstraNav-Memory proposes a fully image-centric long-term memory system, utilizing efficient token compression atop frozen DINOv3 ViT features integrated end-to-end with Qwen2.5-VL-3B navigation policy. This approach advocates implicit memory retention—storing compact representations of hundreds of images within contextual model input, thereby enabling persistent and reusable visual memory across tasks.

Figure 1: Lifelong embodied agent workflow: exploration in unseen environments followed by memory-guided path planning, with persistent environment and agent state across tasks.

Technical Architecture

The core innovation of AstraNav-Memory is the design of a ViT-native visual tokenizer that dramatically reduces context length while retaining navigationally critical spatial-semantic cues. Each first-person image is passed through a frozen DINOv3 ViT backbone with robust spatial and semantic feature extraction. Rather than applying lossy pooling or sparse pruning, AstraNav-Memory utilizes a structured sequence of PixelUnshuffle operations followed by lightweight $3 \times 3$ convolutional blocks (stride 1), incorporating batch normalization and SiLU activation. This technique rearranges the patch dimension and compresses feature grids by up to $16\times$, yielding a per-frame token budget as low as 30 per image—compatible with downstream multimodal model input.

Figure 2: AstraNav-Memory pipeline: DINOv3 ViT features compressed into a compact token stream (from 598 to 30 tokens per image) for low-cost long-horizon reasoning in Qwen2.5-VL-3B.

Positional information is maintained using 2D encoded grids, and all camera poses are serialized as text, under the unified input sequence: $$[\textrm{SYS}; (P_1, I_1); \ldots; (P_T, I_T); \textrm{INSTR}]$$. The model outputs natural language instructions wrapping explicit coordinates for targets, preserving closed-loop autonomy. Critically, the compression blocks are parameter-efficient and frozen DINOv3 prevents co-adaptation between encoder and tokenizer, further aiding drop-in backbone upgrades. The total compute, memory, and attention load are all dramatically reduced, allowing context windows to represent hundreds of sequential observations without modifying subsequent vision-language modules.

Experimental Results

AstraNav-Memory sets new state-of-the-art results on rigorous indoor multi-modal navigation benchmarks including GOAT-Bench and HM3D-OVON. On the most challenging GOAT-Bench "Val-Unseen" split, AstraNav-Memory achieves a 62.7% Success Rate (SR) and 56.9% Success weighted by Path Length (SPL), outperforming previous methods (MTU3D: 47.2% SR, 27.7% SPL) by margins of 15.5% (SR) and 29.2% (SPL). Comparable improvements are observed in familiar environments (Val-Seen, Val-Seen-Synonyms) and open-vocabulary settings (HM3D-OVON), highlighting strong generalization to both novel scenes and novel instruction sets.

Efficiency analysis demonstrates that moderate compression ratios (e.g., 16$\times$) strike an optimal trade-off between context length and performance, supporting visual histories up to 200 frames while maintaining high navigation accuracy and dramatically reducing GPU memory and training/inference latency. Excessive compression ($64\times$) leads to substantial performance drop, evidencing the limits of semantic retention under aggressive reductions.

Ablations and Analysis

Detailed ablations reveal:

• Memory length under fixed compression: Accuracy peaks for 100 compressed frames ($16\times$), suggesting longer histories help up to a point before attention dilution and recency loss degrade performance.
• Compression rate: 4$\times$ and 16$\times$ offer practical trade-offs; higher rates induce feature loss and sharply reduce SR/SPL.
• Backbone size: Larger DINOv3 variants marginally improve success rate, but with significant parameter overhead (ViT-L: 300M vs ViT-B: 86M).
• Visual representation: Category-level breakouts and patch similarity visualizations identify strong object-centric semantic grouping (e.g., bookshelves, appliances) but persistent challenges in texture and boundary-sensitive categories (carpets), motivating integration of segmentation or boundary-aware modules in future work.

  Figure 3: Visualization of DINOv3 patch similarities—semantic grouping succeeds on object-centric targets (left), but fails to clearly distinguish textures and boundaries (right).

  

  Figure 4: GOAT-Bench navigation: Top demonstrates exploration with incremental memory accumulation; bottom shows rapid, memory-guided optimal navigation following subsequent instructions.

Practical and Theoretical Implications

Practically, AstraNav-Memory enables embodied agents to operate in persistent, non-episodic indoor environments where memory is retained across sessions and tasks, supporting efficient multi-goal navigation and real-world deployment scenarios (e.g., household robotics, interactive assistance). The reliance on compressed visual memory rather than explicit mapping pipeline reduces cross-domain brittleness, engineering complexity, and training overhead, and is less restrictive than object-centric retrieval methods. End-to-end coupling with navigation objectives aligns perceptual compression with downstream control utility, not just perception fidelity.

Theoretically, this approach shifts embodied navigation memory from explicit, retrievable entities toward scalable, implicit spatio-temporal representation, aligning with recent advances in world model architectures and long-context VLM/LLM techniques. The observed attention/recency trade-offs, semantic retention under compression, and mid-level spatial representation inspire deeper exploration into structured long-horizon token management, streaming/recurrent memory integration, and multimodal adaptation.

Future Directions

Advancements can include:

• Augmenting compressed image-centric memory with semantic segmentation, boundary-aware modules, or hybrid memory architectures (combining implicit and explicit retrieval mechanisms).
• Extending token compression for continuous video, 3D semantic mapping, and open-world navigation tasks.
• Exploring dynamic context-resizing and adaptive attention mechanisms for extremely long-horizon reasoning.
• Integrating interactive feedback, test-time adaptation, and memory update routines for persistent adaptation under distributional shift.

Conclusion

AstraNav-Memory introduces a highly efficient, plug-and-play image-centric memory system for lifelong embodied navigation, leveraging structured token compression to enable massive context capacity in vision-LLMs. By shifting the paradigm from brittle object-centric pipelines to compressed, end-to-end implicit memory, AstraNav-Memory achieves state-of-the-art performance, scalable efficiency, and strong generalization. Persisting limitations in texture/boundary discrimination recommend future research into multimodal compression and context management. The framework provides a robust foundation for deploying real-world embodied agents with persistent, adaptive, and scalable memory (2512.21627).