FSUNav: A Cerebrum-Cerebellum Architecture for Fast, Safe, and Universal Zero-Shot Goal-Oriented Navigation

Abstract: Current vision-language navigation methods face substantial bottlenecks regarding heterogeneous robot compatibility, real-time performance, and navigation safety. Furthermore, they struggle to support open-vocabulary semantic generalization and multimodal task inputs. To address these challenges, this paper proposes FSUNav: a Cerebrum-Cerebellum architecture for fast, safe, and universal zero-shot goal-oriented navigation, which innovatively integrates vision-LLMs (VLMs) with the proposed architecture. The cerebellum module, a high-frequency end-to-end module, develops a universal local planner based on deep reinforcement learning, enabling unified navigation across heterogeneous platforms (e.g., humanoid, quadruped, wheeled robots) to improve navigation efficiency while significantly reducing collision risk. The cerebrum module constructs a three-layer reasoning model and leverages VLMs to build an end-to-end detection and verification mechanism, enabling zero-shot open-vocabulary goal navigation without predefined IDs and improving task success rates in both simulation and real-world environments. Additionally, the framework supports multimodal inputs (e.g., text, target descriptions, and images), further enhancing generalization, real-time performance, safety, and robustness. Experimental results on MP3D, HM3D, and OVON benchmarks demonstrate that FSUNav achieves state-of-the-art performance on object, instance image, and task navigation, significantly outperforming existing methods. Real-world deployments on diverse robotic platforms further validate its robustness and practical applicability.

• The paper introduces a dual-brain architecture that combines high-level semantic reasoning with low-level real-time control to achieve fast, safe, and universal zero-shot navigation.
• It integrates VLM-based semantic processing with DRL-driven obstacle avoidance, outperforming state-of-the-art methods on metrics such as SR and SPL in simulation experiments.
• The system demonstrates robust cross-embodiment adaptability in both simulated and real-world deployments, enabling platform-agnostic performance across diverse robotic systems.

FSUNav: A Cerebrum–Cerebellum Architecture for Fast, Safe, and Universal Zero-Shot Goal-Oriented Navigation

Introduction and Motivation

The FSUNav framework presents a dual-brain collaborative architecture designed to address persistent limitations in vision-language navigation (VLN), particularly regarding cross-embodiment transfer, real-time safe deployment, and open-vocabulary semantic adaptation. Existing VLN approaches often exhibit strong dependence on specific robot morphologies, insufficient real-time performance, limited safety mechanisms, inadequate semantic generalization beyond predefined categories, and poor support for multimodal task inputs. FSUNav's solution comprises two principal modules: the Cerebrum (high-level semantic and spatial reasoning) and the Cerebellum (low-level, real-time motion control), working synergistically to enable platform-agnostic, efficient, and safe navigation through open-vocabulary, multi-modal goal specifications.

Figure 1: The FSUNav framework, featuring dual Cerebrum–Cerebellum modules to enable fast, safe, and generalizable zero-shot vision-language navigation.

Architectural Overview

Dual-Brain Design

The architecture employs a modular separation:

• Cerebrum Module: Implements high-level scene understanding, multimodal goal parsing, and target-grounded reasoning using a powerful vision-LLM (VLM) orchestrated via semantic, spatial, and rule-based layers.
• Cerebellum Module: Provides local, geometry-aware navigation and real-time obstacle avoidance through deep reinforcement learning (DRL), abstracted from platform specifics.

Cerebrum Module: Unified Multi-Layer Reasoning

The Cerebrum’s hierarchical reasoning is articulated via:

• Semantic Layer: Employs a VLM to parse goals (object category, textual instruction, image reference) into a structured representation, enabling open-vocabulary grounding without reliance on predefined detectors.
• Spatial Layer: Maintains a BEV geometric map, integrating VLM-driven semantic waypoints and geometry-based frontier exploration for efficient trajectory proposals under both observed and occluded goals.
• Rule Layer: Coordinates behavior, alternating between semantic exploration and robust two-stage VLM-based verification for target identification and localization. An adaptive cooldown mechanism mitigates computational overhead from frequent semantic queries, while scene graph construction enriches higher-level reasoning.

  Figure 2: FSUNav_Cerebrum: Layerwise design enabling training-free, open-vocabulary, and multimodal task adaptation across navigation domains.

Cerebellum Module: Platform-Agnostic RL Control

The Cerebellum module applies a DRL policy (SAC-based), where platform abstraction is achieved through:

• Dimension-configurable input representation encompassing robot size and dynamic constraints, processed by a PointNet backbone for variable sensor and embodiment generalization.
• Curriculum learning over a continuous morphology space to train adaptive and robust collision-avoiding control, enabling seamless cross-deployment on wheeled, quadrupedal, and humanoid platforms.

Experimental Results

Simulation-Based Evaluation

FSUNav_Cerebrum was evaluated on MP3D and HM3D across ObjectNav, Instance-ImageNav, TextNav, and Open-Vocabulary tasks using standardized SR and SPL metrics. Across all tasks, FSUNav_Cerebrum achieved the highest reported SR and SPL values, outperforming both supervised and training-free baselines, including ZSON, PSL, GOAT, ESC, VLFM, SG-Nav, CogNav, REST, and UniGoal. Notable advances include:

• ObjectNav (HM3D): 76.2% SR, 40.49% SPL (vs. CogNav’s 72.5%, 26.2% and PEANUT+LOG’s 64.3%, 34.2%).
• Instance-ImageNav (HM3D): 72.6% SR, 34.88% SPL (vs. IEVE’s 70.2%, 25.2% and UniGoal’s 60.2%, 23.7%).
• Text-Nav (HM3D): 39.8% SR, 21.58% SPL (vs. UniGoal’s 20.2%, 11.4%).
• Open-Vocabulary Nav (HM3D): 56.87% SR, 32.59% SPL, substantially above MTU3D (40.8%, 12.1%) and VISOR (28.48%, 17.26%).

These results constitute strong evidence for the claim that FSUNav achieves universal, training-free, task and platform-agnostic navigation with leading efficiency and robustness (2604.03139).

Real-World Robotic Deployment

The system was deployed on a Unitree Go2 EDU quadruped, equipped with an Intel RealSense D455 (RGB) and Livox MID-360 LiDAR for robust environment mapping and real-time collision avoidance. The local planner operates solely with geometric information, and the perception module demonstrates effective estimation using only RGB-derived depth, eliminating specialized sensor dependencies.

Figure 3: Real-world robot platform configuration for FSUNav experimentation.

Experiments validated that FSUNav supports high-speed navigation (0.6 m/s) with dynamic avoidance and accurate real-world object identification in unstructured environments, under open-vocabulary and multimodal goal inputs. The system's closed-loop safety and adaptability were maintained without task-specific retraining or prior knowledge of the operational scenes.

Figure 4: The robot executes open-vocabulary navigation ('umbrella') with real-time obstacle avoidance, robust object approach, and accurate identification in unstructured real-world settings.

Analysis and Implications

Advancements over Prior Art

FSUNav demonstrates several distinct advantages:

• Zero-shot universality: Training-free operation across ObjectNav, Instance-ImageNav, TextNav, and Open-Vocabulary tasks.
• Cross-embodiment generalization: Platform-agnostic deployment, validated across diverse real and simulated morphologies.
• Integrated semantic reasoning: End-to-end use of VLMs across detection, semantic exploration, verification, and scene graph construction, supplanting piecemeal pipelines.
• Multimodal interaction: Unified handling of text, image, and category goals with no architecture or training changes.
• Real-time safety: Explicit geometric embedding and DRL-based collision avoidance yielding efficient, risk-mitigating control.

Theoretical and Practical Significance

Practically, FSUNav provides a deployable solution for universal VLN, circumventing requirements for task or platform-specific data, retraining, or architectural reconfiguration. Theoretically, it demonstrates that VLM-centric reasoning, combined with appropriate geometric and RL-based control abstractions, is sufficient for robust open-world generalization, enabling future research to focus on progressive cognitive layering atop a shared reasoning and execution substrate.

Potential future directions include automated scene graph-driven high-level reasoning, transfer to hybrid indoor-outdoor domains, deeper integration with human-in-the-loop interaction, and full integration with emerging robotics foundation models for broader embodied AI.

Conclusion

FSUNav introduces an effective dual-brain architecture, supporting fast, safe, and universal zero-shot navigation with open-vocabulary, multimodal task specification. Through VLM-driven hierarchical reasoning, platform-agnostic RL-based control, and comprehensive empirical validation, FSUNav sets a new state-of-the-art in training-free universal VLN for heterogeneous robotic systems. Its robust real-world performance and flexible extensibility position it as a strong foundation for further advancements in embodied AI (2604.03139).