NORM-Nav: Zero-Shot Norm-Aware Navigation

• NORM-Nav is a zero-shot mobile robot navigation framework that integrates natural language instructions into costmap-based planning for human-centered environments.
• It employs a two-stage process using LLM-based constraint parsing and vision–LiDAR fusion to ground behavioral instructions in real-time scenes.
• The framework fuses multi-layer costmaps—geometric, semantic, directional, and velocity—to guide trajectories while ensuring reliability and adherence to human norms.

Searching arXiv for NORM-Nav and closely related norm-aware navigation work. NORM-Nav is a zero-shot mobile robot navigation framework that integrates natural language behavioral constraints into costmap-based planning. It is designed for human-centered environments in which collision-free motion is insufficient because trajectories must also follow local behavioral conventions such as keeping to the right side of a road, slowing near people or cars, taking a preferred side around an obstacle, or traversing semantically traversable entities such as curtains. Its central design choice is to preserve the reliability and efficiency of classical planning while augmenting it with language-parsed, perception-grounded behavioral constraints encoded as multi-layer costmaps that are directly compatible with standard grid-based planners (Huo et al., 16 May 2026).

1. Problem formulation and motivation

NORM-Nav addresses path planning under natural language behavioral constraints with real-time perception. The robot is given a start $S$, a goal $T$, a stream of onboard observations $\{O_t\}$ from LiDAR and camera, and free-form behavioral instructions $C$. The planning objective is formulated as

$$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$

where $P$ is the set of feasible trajectories consistent with robot dynamics and the observed environment, $J(T)$ is the trajectory cost, and $T=C$ means that the trajectory satisfies the interpreted and grounded behavioral constraints (Huo et al., 16 May 2026).

The framework is motivated by a limitation of conventional costmap-based navigation. Standard costmaps emphasize collision avoidance and geometric feasibility, but they do not by themselves encode semantic distinctions, side preferences, or natural-language speed regulations. The paper identifies several concrete failures of this paradigm. A grassy patch, sidewalk, curtain, manhole cover, or crosswalk may be traversable or non-traversable depending on context; classical costmaps cannot encode instructions such as “keep right,” “pass on the left,” or “stay near the center”; LiDAR-only systems may misclassify curtains as rigid obstacles; and standard planners do not naturally interpret commands such as “move quickly across the crosswalk” or “slow down near the car” (Huo et al., 16 May 2026).

A common misconception is to treat NORM-Nav as an end-to-end learned navigation policy. It is not. The system does not replace the planner with a learned controller; instead, it augments classical planning with structured, language-derived costs. That architectural choice is central to the method’s zero-shot positioning.

2. Constraint parsing and behavioral representation

The first stage uses an LLM to parse free-form natural-language instructions into structured behavioral constraints. NORM-Nav distinguishes between offline constraints $L_{\text{off}}$, which are global and task-wide, and online constraints $L_{\text{on}}$, which are short-term and situational. They are merged as

$T$0

If online and offline instructions conflict, online constraints take precedence (Huo et al., 16 May 2026).

The parsed instruction set is represented as

$T$1

where $T$2 is the referenced object, $T$3 is the directional preference, $T$4 is the velocity requirement, and $T$5 is the traversability constraint. Representative values reported in the paper include objects such as pedestrian, vehicle, grass, crosswalk, curtain, and manhole cover; $T$6; $T$7; and $T$8 (Huo et al., 16 May 2026).

The paper’s example makes the representation explicit. Given the offline instruction “Walk quickly while crossing the crosswalk” and the online instruction “Please walk on the right side of the crosswalk, then go around the car from the left side. When passing the car, walk slowly,” the parser yields:

• $T$9
• $\{O_t\}$0

This schema is more restrictive than free-form prompting but more expressive than a pure obstacle map. It decomposes behavioral language into object references and three executable dimensions: directional bias, speed preference, and traversability.

3. Perception grounding with vision–LiDAR fusion

The second stage grounds the structured constraints in the current scene using camera–LiDAR fusion. For open-vocabulary semantic grounding, at time $\{O_t\}$1, the method takes an RGB image $\{O_t\}$2 and an object text $\{O_t\}$3, and uses GSAM2 to produce a binary mask $\{O_t\}$4. The mask pixel set is

$\{O_t\}$5

The LiDAR point cloud is

$\{O_t\}$6

Each LiDAR point is projected into the image plane; if the projected pixel lies inside $\{O_t\}$7, the point is associated with object $\{O_t\}$8, yielding the object-specific subset $\{O_t\}$9 (Huo et al., 16 May 2026).

To improve robustness, the object point cloud is denoised with DBSCAN and accumulated over a sliding window: $C$0 On the bird’s-eye-view plane, each grounded object is then approximated by an axis-aligned bounding box

$C$1

This $C$2 is the geometric anchor for subsequent costmap construction (Huo et al., 16 May 2026).

The grounding stage is operationally important because the language representation is object-centric. Commands such as “go through the curtain,” “pass the car on the left,” or “keep right on the crosswalk” are only executable once the corresponding entities have been localized in the current scene. The method therefore treats grounding not as a generic detection problem but as the interface between semantic reference and planner-compatible spatial structure.

4. Multi-layer costmaps and planner integration

NORM-Nav constructs four cost layers: a geometric traversability layer, a semantic traversability layer, a directional constraint layer, and a velocity constraint layer. These are fused into a spatial behavioral costmap $C$3 and a kinematic behavioral costmap $C$4 (Huo et al., 16 May 2026).

Layer	Encoded signal	Operational role
Geometric traversability	LiDAR occupancy and obstacle geometry	Baseline collision avoidance
Semantic traversability	Traversable vs non-traversable object semantics	Corrects geometric interpretation
Directional constraint	left, right, middle preferences	Shapes side-specific path bias
Velocity constraint	slow, normal, fast requirements	Regulates local translational velocity

The geometric traversability layer is a standard occupancy-style costmap from LiDAR geometry. After ground and height filtering, obstacles are defined as

$C$5

Projected to the bird’s-eye-view grid, occupied cells receive high cost, providing the baseline collision-avoidance substrate (Huo et al., 16 May 2026).

The semantic traversability layer corrects that substrate using object semantics and explicit instruction. For an object region $C$6, the mean geometric cost is

$C$7

If traversability is not explicitly specified, it is inferred from geometry: $C$8 and the semantic cost map is

$C$9

In the paper’s interpretation, this allows semantic knowledge to override raw LiDAR beliefs, for example by treating curtains as traversable or grass as non-traversable (Huo et al., 16 May 2026).

The directional constraint layer encodes instructions such as “pass the car on the left,” “keep to the right side of the road,” or “go around the manhole cover from the left.” If a direction $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$0 is specified, the object box may be expanded when $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$1 is non-traversable, and a lateral cost profile is assigned. Table II in the paper specifies the boundary pattern:

• left: $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$2
• right: $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$3
• middle: $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$4

The interpolated directional cost is

$$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$5

Here $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$6 controls the sharpness of the preference gradient; the paper’s ablation later studies this parameter directly (Huo et al., 16 May 2026).

The velocity constraint layer encodes speed requirements. If $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$7, a local velocity cost is assigned within $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$8, with distinct handling for slow, normal, and fast. In the authors’ description, this layer modulates local translational velocity where the robot is currently located, so that the planner can slow down or speed up according to linguistic context (Huo et al., 16 May 2026).

The fused costmaps are then inserted into an otherwise standard navigation stack. NORM-Nav uses A* as the global planner and TEB as the local planner, with no change to the underlying planner. The spatial map determines where to go; the kinematic map determines how fast to move there. This separation is one of the framework’s defining features.

5. Experimental configuration and empirical results

The simulation setup uses the MEDIUM environment from OpenBench and a four-wheeled differential-drive platform. The real-world setup uses a SCOUT 2.0 differential-drive mobile robot equipped with a Livox MID-360 LiDAR and an Intel RealSense D455 depth camera. Onboard compute consists of an AMD R9-7945HX CPU and an NVIDIA RTX 4060 GPU, and the LiDAR is tilted by 20 degrees to scan the ground. Implementation details include Qwen-VL-Max as the instruction parser, $$T^* = \arg\min_{T \in P} J(T) \quad \text{subject to } T = C,$$9, $P$0, $P$1, costmap resolution 0.05 m, planning horizon $P$2, A* as the global planner, and TEB as the local planner (Huo et al., 16 May 2026).

The baselines are BehAV, InstructNav, and INF. Evaluation uses four metrics: Success Rate (SR), Success weighted by Path Length (SPL), Fréchet Distance (FD), and Behavioral Following Accuracy (BFA). Tasks are grouped into four types: Region-Following, Region-Avoidance, Traversable-Obstacle, and Combined Task (Huo et al., 16 May 2026).

The reported quantitative results place NORM-Nav ahead of the baselines across all four task categories. In simulation:

• Region-Following: NORM-Nav achieves SR 90%, SPL 65.77%, FD 2.14, and BFA 89%. BehAV reports SR 60%, SPL 19.63%, FD 4.21, and BFA 62%; InstructNav reports SR 50%, SPL 15.14%, FD 5.36, and BFA 58%.
• Region-Avoidance: NORM-Nav achieves SR 90%, SPL 58.92%, FD 2.16, and BFA 87%, compared with BehAV at SR 60%, SPL 38.34%, FD 2.82, BFA 64%, and InstructNav at SR 40%, SPL 17.68%, FD 5.10, BFA 61%.
• Traversable-Obstacle: NORM-Nav achieves SR 80%, SPL 67.26%, FD 1.35, and BFA 84%. BehAV reports SR 70%, SPL 52.14%, FD 2.23, and BFA 67%. The paper states that INF handles traversability better than InstructNav, but lacks behavior awareness.
• Combined Task: NORM-Nav achieves SR 90%, SPL 54.87%, FD 3.01, and BFA 85%. BehAV reports SR 30%, SPL 18.62%, FD 7.02, and BFA 39%; InstructNav reports SR 20%, SPL 12.21%, FD 6.45, and BFA 36%; INF fails to handle the combined setting (Huo et al., 16 May 2026).

These metrics support two distinct interpretations. First, the higher SR and SPL indicate that the framework is not only behaviorally expressive but operationally successful. Second, the lower FD and higher BFA indicate that the executed trajectories are closer to human teleoperated references and more consistent with the instruction semantics. The paper also reports qualitative behaviors including keeping to the correct side of crosswalks and roads, slowing near cars when instructed, traversing curtains that LiDAR may misclassify as obstacles, and avoiding or biasing around manhole covers according to side-specific instructions (Huo et al., 16 May 2026).

6. Parameterization, limitations, and place within norm-aware navigation

The principal parameter study concerns the interpolation parameter $P$3 in the directional layer. The paper reports that $P$4 leads to weak adherence to directional preference, $P$5 yields moderate adherence, and $P$6 produces stronger compliance with user instructions. The authors conclude that $P$7 provides the best balance between safety and compliance, and the implementation uses $P$8 (Huo et al., 16 May 2026).

The paper does not present an extended failure-analysis section, but several limitations are explicit or strongly implied. The method depends on correct grounding: if the vision-language component misidentifies the referenced object, the wrong constraint is encoded. LLM parsing errors can mis-structure ambiguous instructions into $P$9. Costmap discretization can be coarse for very fine behavioral nuance. Real-time performance depends on stable camera–LiDAR fusion and robust object masks. The framework is also primarily object-centric, so very complex multi-agent social reasoning is not its core focus (Huo et al., 16 May 2026).

Within the broader literature on norm-aware navigation, NORM-Nav occupies a specific design point. It operationalizes norms as planner-compatible costmaps rather than as direct action generation. Contemporary systems make different commitments. LISN defines language-instructed social navigation as an instruction-conditioned problem and uses a fast-slow hierarchy in which a VLM modulates costmaps and controller parameters (Chen et al., 10 Dec 2025). NaviWM augments LLM reasoning with a spatial-temporal world model and a deductive reasoning module in which social norms are encoded as first-order logic and processed through a Gentzen-style natural deduction tree (Wang et al., 27 Oct 2025). OLiVia-Nav distills the social reasoning capabilities of a large VLM into a lightweight VLM and couples that representation to trajectory prediction, trajectory selection, and online lifelong updates (Narasimhan et al., 2024). This suggests a broader methodological split: some systems embed norms in symbolic or linguistic reasoning layers, whereas NORM-Nav embeds them in the planner’s cost geometry.

That distinction clarifies the significance of NORM-Nav. Its contribution is not merely that it handles behavioral instructions, but that it shows how natural-language norms can be converted into geometric, semantic, directional, and velocity cues without abandoning the classical planning stack. In that sense, NORM-Nav is a planner-centric account of norm-aware navigation: language is parsed, perception grounds the referents, costmaps encode the constraints, and standard planners remain the execution substrate.