A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation

Abstract: Navigation is a fundamental capability of autonomous systems, yet most existing approaches rely on highly structured models and strong prior assumptions, limiting their robustness in open and uncertain real-world environments. Vision-and-Language Navigation (VLN) offers a promising direction by enabling robots to integrate natural language understanding with visual perception in a data-driven manner. Although VLN has attracted increasing research attention, systematic methodological taxonomy and real-world validation remain limited. This survey presents a comprehensive review of VLN research. Specifically, state-of-the-art methods are organized along two orthogonal dimensions: action paradigms, including hierarchical and monolithic frameworks, and model paradigms, including discriminative and generative approaches. A critical analysis of their respective strengths and limitations is provided. Additionally, we conduct a systematic real-world evaluation of representative VLN system configurations on a physical robotic platform. Experiments across ten diverse real-world scenes show a substantial performance gap between simulation and real-world deployment under the tested configurations: a representative monolithic RGB-only method achieves 61% success in simulation but drops to 22% in real-world deployment, while a hierarchical framework achieves a higher real-world success rate of 51%, suggesting stronger robustness in our evaluation setting. Finally, we highlight key challenges in perception, decision-making, and control that must be addressed in future research.

• The paper presents a comprehensive survey of embodied VLN, categorizing methods into hierarchical and monolithic frameworks to expose the simulation-to-real gap.
• It systematically evaluates approaches using real-world experiments, highlighting challenges in explicit spatial memory and intent grounding.
• Real-world tests reveal that hierarchical methods achieve lower collision rates (7%) and higher success (51%) compared to monolithic models (51% CR, 22% success).

Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation

Introduction and Context

The paper "A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation" (2607.09792) presents an authoritative, systematic synthesis of Vision-and-Language Navigation (VLN), an embodied AI paradigm at the intersection of computer vision, natural language processing, and robotics. VLN tasks require agents to interpret free-form language instructions and navigate in open, unstructured, and previously unseen environments using only egocentric visual perceptions, often with no prior map access or global localization. This work addresses both the breadth of methodological advances and the gap between high simulation performance and real-world deployability.

Figure 1: Overview of the vision-and-language navigation (VLN) research.

The survey encompasses taxonomic categorization, problem formalization, paradigm evolution (hierarchical vs. monolithic frameworks; discriminative vs. generative modeling), dataset and simulation infrastructure, and a systematic, multi-environment real-world empirical evaluation. The latter enables quantification of the simulation-to-real gap and diagnosis of embodied failures beyond the reach of prior purely simulated studies.

Evolution of VLN: Problem Definition and Methodological Taxonomy

VLN originated from the need to transcend the brittleness of classical, model-driven navigation techniques anchored by SLAM and rule-based planning. Early efforts in VLN leveraged deep learning, panoramic sensors, and imitation learning in discretized navigation graphs; subsequent research shifted toward end-to-end continuous control and, more recently, large foundation models capable of reasoning, abstraction, and transfer.

Figure 2: Trend of vision-and-language navigation (VLN) publications from 2018 to 2025.

Figure 3: Survey structure—origins, paradigm classification, frameworks, real-world experiments, and future directions.

The field is now organized primarily along two orthogonal axes:

• Action Paradigm: Hierarchical frameworks (waypoint-based, high-level planners plus low-level controllers) vs. monolithic frameworks (end-to-end action policies).
• Model Paradigm: Discriminative models (action selection or regression) vs. generative models (autoregressive, open-ended reasoning, typically leveraging foundation LLMs/VLMs).

This evolving taxonomy is visualized in Figure 4, which also tracks methodological transitions and quantitative shifts.

Figure 4: Timeline, paradigm quadrants, and evolution of representative VLN methods.

Hierarchical Frameworks: Structured Planning and Robustness

Hierarchical architectures decompose navigation into global reasoning (waypoint or subgoal selection) and local execution (motion control). The discriminative paradigm dominates legacy work due to data efficiency and modularity, but at the expense of error accumulation, reliance on panoramic viewpoints, and limited flexibility for continuous, sensor-constrained settings.

Figure 5: Structure of the hierarchical framework section.

Figure 6: Representative hierarchical waypoint-based VLN methods—topological graphs, metric maps, and LVLM-infused policies.

Recent advances incorporate LVLMs or LLM-based modules for open-ended plan synthesis and zero-shot or prompt-engineered navigation, showing promise for generalization and long-horizon apoptotic tasks.

Key observations:

• Explicit spatial memory (topological/metric maps)—either as structured tokens or as BEV/semantic grids—improves memory, error recovery, and effective reasoning in large environments.
• The best simulation performance (e.g., >85% SR in R2R unseen splits) has been achieved when such structures are married to robust cross-modal alignment and large-scale pretraining.

Monolithic Frameworks: End-to-End Learning and Generalist Trends

Monolithic frameworks eschew explicit planning in favor of direct mappings from observations and instructions to low-level actions. Initially, discriminative policies (classification/regression over primitive moves) prevailed but struggled with long-horizon memory and environmental uncertainty.

Figure 7: Structure of the monolithic framework section.

Figure 8: Representative monolithic methods—recurrent, video-language, and 3D-grounded policies.

Emerging approaches leverage generative LLMs, LVLMs, and video-based foundation models to predict action sequences conditioned on long-horizon state, leveraging memory caches, implicit geometric reasoning, or autoregressive policy heads. These models exhibit substantially broader inductive capacities and cross-task generalizability, as evidenced by unified navigation–QA–manipulation architectures.

Limitations persist:

• Monolithic frameworks, especially those that rely only on monocular RGB, are empirically fragile in real-world settings: insufficient obstacle awareness leads to elevated collision rates and reduced task completion, despite strong simulation metrics.

Real-World Systematic Evaluation

The authors implement a rigorous, empirical evaluation of two state-of-the-art VLN approaches—a panoramic+SLAM-enabled hierarchical framework (CLASH) and a monocular RGB-only monolithic framework (JanusVLN)—on a wheeled robot platform.

Figure 9: Wheeled robot platform used for real-world VLN evaluation.

Ten real-world environments spanning diverse indoor and outdoor scenes are selected Figure 10, and varied navigation instructions (step-by-step, intention/ambiguous, and backtracking) are issued, totaling 200 episodes Figure 11.

Figure 10: Real-world evaluation environments.

Figure 11: Example natural language instructions and navigation trajectories.

A notable visual domain shift between simulated (Habitat) and real-world scenes is highlighted Figure 12.

Figure 12: Simulation (Habitat) vs. real-world visual observations.

Key quantitative results:

• The hierarchical framework achieves 51% SR/37% SSR/67% OSR, while monolithic achieves only 22% SR/17% SSR/27% OSR in real-world deployments under tested conditions.
• The collision rate (CR) for the monolithic approach is 51%, while hierarchical is only 7%, indicating a severe shortfall in safe execution for monocular monolithic policies Figure 13.

Figure 13: System-level comparison of hierarchical and monolithic VLN frameworks in real-world deployment.

Scene-level breakdown confirms consistent outperformance of hierarchical methods across physical settings, except in rare scene-specific exceptions tied to sensor limitations Figure 14.

Figure 14: Per-environment success and collision rates for both frameworks.

On intention and backtracking task types, performance drops for both paradigms, with hierarchical outperforming on intention but underperforming monolithic on backtracking Figure 15. A central failure mode is inability to issue semantically correct STOP actions in the absence of explicit cues.

Figure 15: Performance on intention inference and backtracking navigation tasks.

A detailed analysis of step length and inference times Figure 16 underscores that hierarchical approaches are more computationally efficient in total for long tasks, despite higher latency per action, particularly when leveraging large models offloaded to server/cloud infrastructure.

Figure 16: Inference latency and decision step count for both framework types.

Qualitative trajectory visualizations Figure 17 reveal that hierarchical agents better exploit panoramic information without unnecessary in-place rotations, while monolithic agents attempt coarse “look-around” behaviors to compensate for perceptual deficiencies.

Figure 17: Example real-world navigation trajectories by hierarchical and monolithic frameworks.

Collision failure analysis Figure 18 contextualizes persistent limitations in both accessibility of geometric cues (monolithic) and sensor field-of-view constraints (e.g., LiDAR height and SLAM limitations for hierarchical methods).

Figure 18: Typical real-world collision cases observed during deployment.

Limitations and Research Directions

The authors systematically identify current deficiencies and propose future research axes:

Figure 19: Outstanding challenges and proposed future directions for VLN research.

Collision Avoidance and Geometric Reasoning: Purely learning-based agents, particularly monocular models, exhibit dangerously high collision rates when deployed physically; real-world deployment requires the development of persistent obstacle-aware memory, explicit incorporation of low-level kinematic constraints, and risk-sensitive planning. The static world assumption endemic to simulators does not reflect real human environments.

Semantic Stopping and Intention Understanding: Failures in semantic intent grounding and stopping behavior are pervasive, especially with ambiguous or naturally underspecified instructions. This challenge is even more pronounced when navigation must act as a subroutine within manipulation/fetch-and-carry pipelines.

Sim-to-Real Transfer and Generalization: Visual, physical, and instruction domain gaps remain unresolved—despite large-scale pretraining—and performance does not transfer well from simulation to real deployment, even in environments resembling simulation domains.

World Modeling and Causal Reasoning: Embodied agents, unlike humans, lack mechanisms for counterfactual imagination or causal intervention, constraining adaptivity to dynamic, unforeseen hazards.

Data Engines and Unified Environments: Scaling beyond static, photorealistic 3D scans to fully generative, simulatable, and physically interactive learning environments (e.g., via 3D Gaussian Splatting) will be pivotal for next-generation data coverage Figure 20.

Figure 20: Long-term evolutionary trajectories in VLN research: progression toward open-world, generalist embodied agents.

Conclusion

This paper provides a comprehensive, methodical review and empirical analysis of VLN, pushing beyond benchmark-driven simulation to expose the fragility of contemporary systems in the real world. The hierarchical–monolithic and discriminative–generative dichotomies—along with fine-grained empirical evidence on sim-to-real transfer—present a nuanced perspective on what matters for practical embodiment.

Notably, the paper's strong numerical results reveal a substantial sim-to-real gap (e.g., a monolithic approach with 61% simulation SR achieves only 22% in the real world, while hierarchical achieves 51% under the same settings) and make explicit the need for robust, multimodal, and safety-aware memory and reasoning. These findings challenge several prevailing assumptions in the field, especially regarding the reliability of monocular end-to-end and foundation model-based approaches when deployed outside simulation.

Key theoretical implications include the demand for new world modeling frameworks, persistent memory, intent forecasting, and safety under real-world constraints. Practically, the deployment of VLN agents will increasingly depend on advances in sensor fusion, reliable semantic grounding, explicit physical modeling, and data generation engines enabling robust, scalable training for open worlds.

This survey establishes clear baselines and a research agenda toward trustworthy, generalist, and physically robust embodied navigation under real-world uncertainty.