Walk With Me: Long-Horizon Social Navigation for Human-Centric Outdoor Assistance

Abstract: Assisting humans in open-world outdoor environments requires robots to translate high-level natural-language intentions into safe, long-horizon, and socially compliant navigation behavior. Existing map-based methods rely on costly pre-built HD maps, while learning-based policies are mostly limited to indoor and short-horizon settings. To bridge this gap, we propose Walk with Me, a map-free framework for long-horizon social navigation from high-level human instructions. Walk with Me leverages GPS context and lightweight candidate points-of-interest from a public map API for semantic destination grounding and waypoint proposal. A High-Level Vision-LLM grounds abstract instructions into concrete destinations and plans coarse waypoint sequences. During execution, an observation-aware routing mechanism determines whether the Low-Level Vision-Language-Action policy can handle the current situation or whether explicit safety reasoning from the High-Level VLM is needed. Routine segments are executed by the Low-Level VLA, while complex situations such as crowded crossings trigger high-level reasoning and stop-and-wait behavior when unsafe. By combining semantic intent grounding, map-free long-horizon planning, safety-aware reasoning, and low-level action generation, Walk with Me enables practical outdoor social navigation for human-centric assistance.

• The paper proposes a novel map-free framework that translates abstract user intents into long-horizon, socially compliant navigation routes.
• It utilizes a dual-policy system combining high-level VLM semantic planning with low-level VLA for precise, safety-aware trajectory generation.
• Experimental results show a 60% overall success with up to 70% in targeted delivery tasks, highlighting its efficacy and practical limitations.

Map-Free Long-Horizon Human-Centric Social Navigation: Walk With Me

Problem Setting and Motivation

Robotic assistance in open-world outdoor scenarios demands translation of high-level abstract language instructions into actionable, socially compliant, and safety-aware navigation policies. The core challenge is to bridge human intent—such as “I want to go for a walk”—with real-world destinations and to execute navigation over kilometer-scale, dynamic environments populated by pedestrians and diverse traffic participants. Prior work predominantly leverages pre-constructed high-definition (HD) maps or is restricted to short-horizon, indoor, or point-to-point goal navigation, limiting scalability and practical deployment in outdoor, human-centric assistance (Figure 1).

Figure 1: Illustration of the motivation: Walk With Me enables human-centric, HD map-free outdoor navigation tasks like delivery or blind guidance by grounding abstract intent, planning long routes with map priors, and enforcing explicit safety reasoning for compliance.

The presented paper introduces a framework—Walk With Me—that eschews dependence on HD maps. Instead, it fuses public map service APIs, online GPS, and semantic candidate Points of Interest (POIs) with multi-level visual-LLMs (VLMs) and vision-language-action (VLA) policies to conduct real-world, instruction-driven navigation. This system addresses the limitations inherent in prior map-based and end-to-end learning approaches by establishing a hierarchical and adaptable control pipeline robust to long-horizon navigation in unpredictable, socially complex situations.

System Architecture and Methodology

Walk With Me is architected as a hierarchical, closed-loop perception-reasoning-action system. The key novelty lies in decoupling high-level semantic planning from low-level local trajectory generation, connected by adaptive, observation-aware routing and safety assessment. The framework consists of three principal modules (Figure 2):

1. High-Level Semantic Planning: A VLM interprets the abstract, open-vocabulary user instruction alongside GPS context. It retrieves POI candidates via a map API, grounds intent to a discrete destination, and generates a coarse geo-referenced pedestrian route, which is parsed into sequential waypoints.
2. Adaptive Dual-Policy Navigation: At each step—using RGB observations, pose, trajectory history, and active waypoint—the High-Level VLM jointly assesses scene complexity and makes a binary safety decision (proceed vs. stop-and-wait). Routine cases are dispatched to the Low-Level VLA for locally optimal, socially compliant, short-horizon trajectory generation. Complex, ambiguous, or safety-critical situations invoke explicit reasoning and possible halting.
3. Closed-Loop Execution: The agent fuses VLA trajectory output with SLAM-based tracking for robust execution, aligning camera heading, iteratively updating the state, and maintaining loop closure until arrival or safety budget exhaustion.

   Figure 2: Schematic of the Walk With Me framework: high-level instruction is grounded into a route using map APIs; dual VLM/VLA control manages routine progress and escalates complex safety scenes; execution loop continues until destination is reached or failure is determined.

Experimental Results

Evaluation is conducted on a wheeled AGV in unconstrained outdoor environments across last-mile delivery and blind guidance scenarios. The system receives only high-level natural language instructions and must autonomously ground intent, generate routes, and physically navigate with real-time social compliance. Out of 20 diverse real-world trials, Walk With Me achieves an average success rate of 60%. In last-mile delivery tasks (e.g., “Take the milk tea to Building B”), success rises to 70%, while more open-ended blind guidance tasks (e.g., “I want to go shopping”) see 50% success. Failures are predominantly associated with unstructured destination context, dynamic crowds, or ambiguous crossing situations, emphasizing ongoing challenges in semantic grounding and robust safety assessment.

Qualitative observations further showcase system behaviors (Figure 3): the agent can ground an order to the correct building or map a walking request to a park; it demonstrates stop-and-wait compliance at road crossings and adapts trajectories reactively to pedestrian motion.

Figure 3: Visualization of execution: (a) Map-free last-mile delivery with pedestrian avoidance and dynamic rerouting; (b) Blind guidance with open-ended instruction grounding, safety-respecting crossing, and pedestrian-aware navigation.

Ablations demonstrate that the choice of both High-Level VLM and Low-Level VLA backbones exerts substantial impact. Models like MiMo-Embodied and SocialNav-trained policies deliver the best task completion rates, underlining the importance of navigation-oriented model design and adaptation. Comparative VLM analyses reveal that strong general-purpose models may require further adaptation for robust outdoor semantic grounding and safety reasoning.

Social Reasoning and Policy Behaviors

Explicit qualitative analyses of High-Level VLM outputs (Figure 4) demonstrate nuanced scene understanding, particularly in complex environments such as crossings, crowded sidewalks, and instances of pedestrian proximity. The VLM’s ability to integrate visual and contextual cues, and to modulate robot behavior (proceed/wait), is central for achieving robust, human-aligned performance in physically interactive scenes. Nonetheless, there are observed divergences and uncertainties in ambiguous settings, pointing to ongoing limitations in current VLMs’ social reasoning capabilities.

Figure 4: High-Level VLM qualitative safety assessment for crossings, pedestrian proximity, and crowd density, illustrating context-aware stop/go behavior.

At the local level, policy visualizations (Figure 5) indicate that the VLA generates short-horizon, pedestrian- and geometry-aware trajectories, ensuring feasible, socially sensitive progress towards each waypoint. Integration with SLAM and continual re-planning enables adaptation to highly dynamic outdoor environments.

Figure 5: Low-Level VLA policy generates socially compliant, geometry-aware micro-trajectories, adapting at each step based on scene observations and waypoint instructions.

Practical and Theoretical Implications

Practically, Walk With Me demonstrates the feasibility of long-horizon, map-free outdoor navigation with semantic, social, and safety constraints, substantially reducing operational cost and barriers associated with HD mapping while improving generality and scalability. The system directly addresses real-world use cases, such as autonomous delivery and assistive robotics for the visually impaired.

Theoretically, the work reinforces the advantage of hierarchical, modular, and observation-adaptive policy architectures for embodied AI. It highlights the necessity of coupling high-level semantic intent grounding and social reasoning with robust local trajectory policies, each guided by explicit interaction with public environment priors and real-time perception. The findings also expose remaining gaps: current VLMs, while powerful, require further development in fine-grained safety reasoning and context-dependent disambiguation, particularly in edge-case or low-probability scenarios.

Future Directions

Future advances may focus on:

• Enhanced uncertainty modeling for both perceptual inputs and route/POI selection.
• Tightened integration of multi-modal and multi-scale reasoning, potentially involving continuous learning from longitudinally deployed field data.
• Expansion of sensory modalities, including fused audio, depth, or crowd-sourced map corrections, to mitigate failures due to localization or environmental ambiguity.
• Broader generalization to more diverse urban morphologies, rare social configurations, and adaptive interaction modes.

Conclusion

Walk With Me establishes a robust paradigm for HD map-free, human-centric long-horizon outdoor navigation by synthesizing public map priors, VLM-based semantic grounding, safety-aware high-level reasoning, and locally adaptive, socially compliant policy execution. The hierarchical dual-policy design and explicit observation-aware routing mechanisms enable real-world application across multiple challenging scenarios. The framework sets a strong foundation for next-generation assistive robots and points to rich future opportunities in modular embodied AI system design and deployment (2604.26839).