SSR-ZSON: Hierarchical Zero-Shot Object Navigation

• SSR-ZSON is a hierarchical framework that fuses LLM-derived semantic priors with spatial exploration to enable zero-shot object navigation.
• It employs a TARE-based hierarchical approach with region-level and viewpoint-level planners to optimize semantic guidance and spatial coverage.
• SSR-ZSON achieves real-time navigation with significant improvements in success rate and path efficiency over previous methods.

SSR-ZSON (Spatial-Semantic Relation Zero-Shot Object Navigation) is a hierarchical zero-shot object navigation framework that integrates spatial exploration with semantic reasoning to address challenges in navigating unknown environments. Built on the TARE hierarchical exploration paradigm, SSR-ZSON eliminates the requirement for task-specific training or pre-built maps. It efficiently fuses LLM–derived semantic priors with spatial exploration to enable robots to seek arbitrary target objects based on natural-language commands while overcoming insufficient semantic guidance and limited spatial memory. SSR-ZSON achieves real-time operation and delivers quantifiable improvements over prior methods on standard simulation benchmarks and physical platforms (Meng et al., 29 Sep 2025).

1. Foundational Principles and Objectives

SSR-ZSON is designed to address two primary limitations in zero-shot object navigation: (1) inefficient, unguided exploration due to insufficient semantic guidance and (2) agent entrapment resulting from limited spatial memory. The framework’s chief objectives are to:

• Eliminate task-specific training
• Dynamically fuse semantic cues from LLMs with spatial and topological map information
• Overcome prior failure modes by optimizing both spatial coverage and semantic value

Its two core innovations are (1) a Viewpoint Generation Strategy that prioritizes locations offering high semantic density and spatial coverage within traversable sub-regions and (2) an LLM-based Global Guidance Mechanism that continuously evaluates and propagates semantic relevance through both the region-level and viewpoint-level planners.

2. TARE-Based Hierarchical Exploration Architecture

SSR-ZSON adopts and extends the TARE (Task-Aware Region Exploration) framework, structuring exploration into global (region-level) and local (viewpoint-level) planning.

2.1 Region-Level (Global) Planner

• Environment is partitioned into grid sub-regions $C_i$.
• Each sub-region $C_i$ tracks an “unexplored surface coverage” rate and a semantic-association score, the latter derived from LLM outputs.
• Sub-regions whose cumulative score $\bar S_{C_i}$ exceeds a threshold $\tau$ are “activated.”
• The global planner sequences activated sub-regions in descending order of $\bar S_{C_i}$ for navigation prioritization.

2.2 Viewpoint-Level (Local) Planner

• Within a selected region, candidate viewpoints $\{v_j\}$ are densely sampled.
• Each viewpoint is scored using $S_{viewpoint}(v_j)$, which combines geometric coverage and semantic density.
• Viewpoints with highest scores are connected via shortest viable paths to define efficient, high-value exploration trajectories.

Once local exploration in a region is exhausted (semantic and spatial value depleted), the region is marked as “worthless” and the system advances to the next region.

3. Viewpoint Generation and Scoring

Candidate viewpoint selection in SSR-ZSON simultaneously maximizes geometric and semantic value:

• Geometric Coverage Score: For each $v_j$, the number of previously unobserved points $N_{unobserved}(v_j)$ is computed via ray casting or FOV simulation,

$S_{cov}(v_j) = N_{unobserved}(v_j)$

• Semantic Density Score: For all semantic observations $C_i$0 validated by the LLM, within a cylinder of radius $C_i$1 centered at $C_i$2:

$C_i$3

• Combined Viewpoint Score:

$C_i$4

where $C_i$5 are dynamically tuned per scene for optimal tradeoff.

• Locally Smoothed Semantic Relevance: Each $C_i$6 incorporates semantically weighted relevance via Gaussian smoothing of distances to semantic points, generating

$C_i$7

with $C_i$8.

Regional activation then aggregates

$C_i$9

Sub-regions where $\bar S_{C_i}$0 are prioritized.

4. LLM-Based Global Semantic Guidance

SSR-ZSON leverages LLMs for semantic parsing, relevance estimation, and guidance integration across planning hierarchies.

4.1 Instruction Parsing

Free-form user commands are normalized into target classes $\bar S_{C_i}$1 using a prompting (few-shot) strategy.

4.2 Semantic Relevance Evaluation

For each observed instance $\bar S_{C_i}$2 (e.g., “monitor,” “bed”), semantic association with $\bar S_{C_i}$3 is scored by:

1. Scene localization: Mapping to probable room type
2. Topological analysis: Graph connectivity to $\bar S_{C_i}$4
3. Functional coupling: Chain-of-thought templates yielding $\bar S_{C_i}$5

Each score is cached in an LRU structure (24-hour TTL) to reduce redundant queries. Object points receive a semantic weight:

$\bar S_{C_i}$6

with $\bar S_{C_i}$7 as “semantic boost,” and $\bar S_{C_i}$8 to prevent zero influence points.

4.3 Planning Integration

Weighted points affect both $\bar S_{C_i}$9 (viewpoint selection) and $\tau$0, with the global planner prioritizing regions of high aggregate LLM-assigned value.

5. System Implementation and Experimental Setup

SSR-ZSON was evaluated in both simulation and physical deployments:

5.1 Simulation

• Datasets: Matterport3D (MP3D), Habitat-Matterport3D (HM3D).
• Semantic meshes from Habitat environments are converted to point clouds in Gazebo.
• Real-time SLAM achieved with Livox Mid360 LiDAR.
• Semantic object detection via Mask R-CNN or YOLO; semantic points input to the LLM pipeline.

5.2 Physical Platform

• Compute: 12-core ARM Cortex-A78AE CPU, 64 GB RAM, 275 TOPS NPU running Ubuntu 20.04.
• Sensors: Intel RealSense D435i (RGB-D), Livox Mid360 LiDAR, 2D wheel odometry.
• Software: ROS Noetic, cartographer SLAM, SSR-ZSON modules as ROS nodes.

6. Performance Evaluation

SSR-ZSON demonstrates substantial improvements in simulation and real-world settings, evaluated via Success Rate (SR) and Success weighted by Path Length (SPL):

Method	SR_MP3D	SR_HM3D	SPL_MP3D	SPL_HM3D
ZSON [22]	15.3	25.5	0.048	0.126
VoroNav [24]	–	42.0	–	0.260
VLFM [VLFM24]	36.4	52.5	0.175	0.304
InstructNav [24]	–	58.0	–	0.209
UniGoal [25]	41.0	54.5	0.164	0.251
SSR-ZSON	59.5	65.7	0.345	0.391

SSR-ZSON surpasses the best prior by ΔSR = +18.5% / +11.2% and ΔSPL = +0.181 / +0.140 on MP3D/HM3D, respectively.

In real-world trials, SSR-ZSON reduces redundant path length by over 35% compared to frontier-based baselines and demonstrates robust semantic prioritization in office/corridor environments for varied language-specified search tasks.

7. Addressing Semantic and Spatial Memory Limitations

The SSR-ZSON framework persistently scores all visible objects via LLM output and carries these relevance metrics into both region and viewpoint-level planning, enabling the robot to focus on areas likely to contain the queried target. The coverage-aware sub-regions memory model details per-region field-of-view coverage, marking fully explored regions as “worthless,” which prevents redundant exploration and local entrapment. SSR-ZSON demonstrates the effective fusion of LLM-driven semantic reasoning and TARE-based hierarchical exploration, resulting in improved success and efficiency for zero-shot object navigation in both simulated and physical settings (Meng et al., 29 Sep 2025).