NORM-Nav: Zero-Shot Mobile Robot Navigation with Natural Language Behavioral Constraints

Abstract: Mobile robots operating in human-centered environments must generate not only collision-free paths but also trajectories that follow local behavioral conventions. Conventional costmap-based navigation emphasizes geometric feasibility and often overlooks such requirements, which can result in socially inappropriate behaviors. This paper presents NORM-Nav, a zero-shot framework that integrates natural language behavioral constraints into costmap-based planning. An LLM parses each instruction into structured constraints and grounds them using real-time vision--LiDAR perception. These constraints are encoded as multi-layer costmaps that represent geometric, semantic, directional, and velocity cues and are directly compatible with standard grid-based planners. Simulation and real-world experiments indicate that NORM-Nav improves task success rates and produces trajectories closer to human references than representative baselines. The project website is available at https://ei-nav.github.io/NORM-Nav.

• The paper introduces NORM-Nav, a framework that enables robots to navigate by interpreting natural language behavioral constraints, integrating these into costmaps for existing navigation planners.
• It leverages large language models (LLMs) for parsing instructions and vision-LiDAR fusion for semantically grounded perception, enabling precise task execution without retraining.
• NORM-Nav demonstrates high task success and compliance, outperforming traditional systems in region-following and obstacle navigation across simulated and real-world tests.

Integrating Natural Language Behavioral Constraints in Zero-Shot Robot Navigation: An Analysis of NORM-Nav

Introduction

"NORM-Nav: Zero-Shot Mobile Robot Navigation with Natural Language Behavioral Constraints" (2605.16979) addresses the challenge of enabling mobile robots in human-centric environments to generate not only collision-free trajectories but also ones that comply with natural language behavioral specifications. The prevailing costmap-based navigation pipelines, though robust and computationally efficient, are ill-equipped to interpret and execute nuanced, context-specific behavioral rules. The paper introduces NORM-Nav, a framework leveraging LLMs in conjunction with real-time vision-LiDAR perception to parse and ground free-form behavioral constraints, encoding them as multi-layer costmaps for direct integration with standard navigation planners.

Technical Contributions

The NORM-Nav framework consists of several novel components:

• Behavioral Constraint Parsing with LLMs: Natural language instructions—covering both long-term (offline/global) and short-term (online/contextual) behaviors—are parsed by an LLM into unified, structured tuples capturing object references, directional constraints, velocity constraints, and traversability attributes. Priority is given to immediate instructions over long-term norms in the event of conflicts.
• Semantically Grounded Multimodal Perception: Real-time fusion of RGB imagery and LiDAR via open-vocabulary segmentation (using GSAM2) enables semantic object detection and association with parsed constraints. Temporal aggregation and spatial filtering (DBSCAN) ensure robust object modeling even under perception noise.
• Multi-Layer Costmap Construction: Four orthogonal costmap layers encode: (1) geometric traversability; (2) semantic traversability based on natural language priors; (3) directional preferences (e.g., keep to the right); (4) velocity modulation. These layers are fused into two principal costmaps—spatial and kinematic—preserving planner compatibility.
• Plug-in Integration with Classical Planners: The modular design allows the costmaps to be consumed by standard global (A*, RRT) and local (TEB, DWA) planners without altering core algorithms, directly enforcing behavioral constraints during path computation and velocity control.

Costmap Encoding of Behavioral Instructions

Behavioral constraints are spatially encoded by interpolating the cost distribution across object bounding boxes, where the interpolation parameter precisely modulates the degree of behavioral enforcement (e.g., strict lane-keeping vs. flexible navigation). Velocity constraints are mapped to local admissible speeds, enabling context-dependent speed adjustments in execution.

Experimental Validation

Benchmarks and Metrics

The framework is evaluated in both simulation (OpenBench-MEDIUM) and real-world deployments using a SCOUT 2.0 robot with high-resolution vision-LiDAR sensing. Evaluation metrics include:

• Success Rate (SR): Goal reaching with full constraint adherence.
• Success-weighted Path Length (SPL): Efficiency vs. shortest feasible path.
• Fréchet Distance (FD): Deviation from human-operated reference trajectories.
• Behavioral Following Accuracy (BFA): Fraction of the path compliant with all specified constraints.

Comparative Results

NORM-Nav demonstrates consistently superior performance across a variety of task regimes—region-following, region-avoidance, traversable obstacle navigation, and combined scenarios—outperforming reference systems such as BehAV [13], InstructNav [28], and INF [17]. Notably:

• Region-Following Tasks: NORM-Nav achieves 90% SR, 65.77% SPL, FD of 2.14, and 89% BFA, significantly outperforming all baselines.
• Region-Avoidance and Traversable-Obstacle Tasks: NORM-Nav maintains high SR (90%, 80%), while baselines fail to deal with misclassified traversable obstacles or side-specific behavioral instructions.
• Combined Tasks: NORM-Nav shows robustness with 90% SR, 54.87% SPL, and 85% BFA, whereas the best baseline achieves only 30% SR and 39% BFA.

Parameter ablation on the directional interpolation parameter shows a direct correlation between higher values and stricter behavioral conformity, enabling tunable trade-offs between safe clearance and behavioral compliance.

Real-World Demonstrations

Physical tests confirm that the system reliably interprets and executes free-form natural language instructions, e.g., keeping to specified sides, bypassing ambiguous obstacles (curtains, manhole covers), or modulating speed near sensitive objects, often producing trajectories closely aligned with human preferences.

Implications and Future Directions

The results of NORM-Nav challenge the traditional dichotomy between efficiency/robustness in classical planners and flexibility in learning-based methods. By bridging LLM-based instruction parsing and visuo-semantic grounding with costmap-based planning, NORM-Nav eliminates the need for large annotated datasets and retraining, while delivering strong zero-shot generalization to unseen instructions and contexts. The architecture invites several practical and theoretical implications:

• Human-Robot Interaction: Transparent, user-friendly behavioral programming via free-form language enhances usability and deployability in public settings.
• Behavioral Norms Enforcement: Explicit encoding of context-dependent conventions or legal/social norms becomes feasible, which is critical for urban navigation and shared spaces.
• Planner Modularity: The plug-in costmap mechanism ensures future compatibility and scalability with advances in both LLMs and navigation systems.
• Toward Adaptive Social Navigation: Integrating conversational or situational reasoning could further elevate human-likeness in navigation, especially under ambiguous or conflicting instructions.

Open questions include scaling to multi-agent environments, extending to 3D navigation, and evaluating long-term system reliability in diverse, multi-user, or unseen perceptual conditions.

Conclusion

NORM-Nav provides a robust, generalizable framework for integrating free-form natural language behavioral constraints into costmap-based mobile robot navigation (2605.16979). By leveraging LLMs for behavioral parsing and multimodal semantic grounding, and by encoding these constraints within planner-compatible costmaps, the approach achieves high task success, behavioral fidelity, and alignment with human path preferences across simulated and real-world domains. This work marks a substantive advancement in the pursuit of socially compliant and adaptable robot navigation systems that remain practical for real-world deployment.