TANGO: Topometric Navigation Pipeline

• TANGO is an RGB-only, object-level topometric navigation pipeline that fuses global planning via an object graph with local metric control for robust robot motion.
• It features a dual-layer system where a topological planner selects object-level sub-goals and a local controller generates BEV traversability maps for short-range trajectory planning.
• An auto-switch mechanism falls back to RoboHop’s controller when local metric planning fails, ensuring seamless navigation in complex, unstructured environments.

TANGO, short for “Traversability-Aware Navigation with Local Metric Control for Topological Goals,” is an RGB-only, object-level topometric navigation pipeline for robot visual navigation. It combines global topological path planning with local metric trajectory control to enable zero-shot, long-horizon robot navigation without requiring 3D maps or pre-trained controllers. The system navigates toward object-level sub-goals, uses monocular depth and traversability estimation for obstacle-aware local control, and includes an auto-switching mechanism that falls back to a purely topological controller from RoboHop when local bird’s-eye-view planning fails. The method is built from foundational models, which gives it open-set applicability without domain-specific fine-tuning, and it is evaluated in both simulated environments and real-world tests (Podgorski et al., 10 Sep 2025).

1. System organization and operational loop

TANGO is organized as a fusion of two control layers. The global layer performs object-level topological planning over an object-graph, while the local layer performs metric motion control from the current RGB observation. At run time, the robot repeatedly alternates between topological planning and metric control. In the first step, it localizes in an object-level graph and computes which object-segment is the next sub-goal through shortest-path search on the graph. In the second step, it converts the current camera image into a 2D bird’s-eye-view traversability cost map, plans a short trajectory to the sub-goal, and sends yaw and forward commands.

This decomposition is designed around a specific systems objective: long-horizon navigation without globally-consistent 3D maps and without a learned action policy. The global planner provides long-range structure through sub-goals, while the local planner continuously re-estimates short-range traversability from the current RGB frame. When the local planner cannot produce a viable path, TANGO auto-switches to RoboHop’s segment-servoing controller until metric traversability becomes usable again.

The resulting behavior is neither a purely reactive controller nor a conventional mapping-and-planning stack. Instead, it is a topometric pipeline in which object-level topology governs long-range progress and local metric reasoning governs immediate motion.

2. Topometric graph representation and global planning

The global representation is an object-level graph. Each node $n_i \in \mathcal{N}$ is an image segment $M_i$ representing a distinct object or region, extracted by SAM. The edges $\mathcal{E}$ are of two kinds. Intra-image edges connect centroids of segments in the same frame through Delaunay triangulation. Inter-image edges connect the “same” segment across different frames through SuperPoint and LightGlue matching.

For global planning, TANGO assigns integer edge weights:

$$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$

Dijkstra’s algorithm is run from every localized segment to the goal segment. This produces a sub-goal cost mask over all visible segments, from which the visible segment with minimal cost is selected as the next topological sub-goal. The published system uses these purely integer weights for topological planning.

The paper also gives a generalized cost view:

$$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$

where $d_{ij}$ is the Euclidean distance between projected centroids in bird’s-eye view, $T_{ij}$ is the traversability score of the connecting region, and $\alpha,\beta \ge 0$ trade off distance against preferred surface. In the published system, $\alpha=1$ and $\beta=0$, so the implemented global planner remains purely topological (Podgorski et al., 10 Sep 2025).

This graph construction is object-centric rather than occupancy-centric. Its function is to expose long-range connectivity between semantically meaningful regions while avoiding the cost and fragility of maintaining a globally-consistent 3D map.

3. Local metric control, traversability estimation, and fallback behavior

The local controller converts a single RGB frame into a short-horizon traversability-aware motion plan. Depth-Anything predicts per-pixel metric depth $M_i$0 from the current image. SAM segments the image into regions $M_i$1. Each segment crop is embedded into CLIP’s joint image-text space and queried with prompts such as “floor,” “ground,” or “rug,” yielding a binary traversability mask

$M_i$2

The controller then projects each traversable pixel $M_i$3 into robot-centered bird’s-eye-view coordinates $M_i$4. From that projection it builds a 2D cost map by starting from a binary occupancy image, applying a distance transform from obstacle edges, and smoothing with a box filter. The farthest point of the chosen sub-goal segment is projected into bird’s-eye view as $M_i$5, and the controller computes a shortest path

$M_i$6

through the BEV grid using Dijkstra’s algorithm. The first few waypoints are converted into continuous motion through fixed forward speed and yaw steering under a proportional line-following law.

The paper also presents a control-theoretic interpretation in which the local planner solves, over $M_i$7, an optimization that minimizes distance to $M_i$8 together with a smoothness term $M_i$9, subject to robot dynamics $\mathcal{E}$0. This formulation emphasizes that the BEV planner is not only selecting free space but also regularizing steering behavior.

The fallback layer is activated under two conditions: the BEV traversability mask $\mathcal{E}$1 is empty, meaning no contiguous free region leading to the sub-goal is detected, or the BEV Dijkstra search fails to find a path. In either case, control switches to RoboHop’s segment-servoing controller:

$\mathcal{E}$2

with

$\mathcal{E}$3

where $\mathcal{E}$4 is the normalized topological distance of segment $\mathcal{E}$5, $\mathcal{E}$6 is the pixel column of the segment centroid, $\mathcal{E}$7 is the image center column, $\mathcal{E}$8 is image width, $\mathcal{E}$9, and $$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$0. The system stays in this fallback mode until enough traversability reappears.

4. Architectural composition and data flow

The perception and planning stack is built from modular components. SAM, using a ViT-based backbone, provides approximately $$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$1–$$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$2 masks per frame. SuperPoint features and LightGlue perform local matching for segment-tracklet association in the graph. Depth-Anything, described as a MiDaS-style encoder-decoder, outputs a $$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$3 metric depth map. The traversability classifier uses CLIP image features and text features for “floor,” “ground,” and “rug,” with nearest-neighbor scoring.

The global planner is a lightweight Dijkstra search on a sparse graph, with $$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$4. The local planner discretizes bird’s-eye view at approximately $$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$5 cm resolution and runs Dijkstra on a $$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$6 grid, with reported runtime of $$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$7–$$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$8 ms per frame. The control loop is ROS-style, runs at $$w(n_i \to n_j)= \begin{cases} 1 & \text{if intra-image} \ 0 & \text{if inter-image (same object instance)} \end{cases}$$9 Hz, and publishes geometry_msgs/Twist messages with linear $$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$0 and angular $$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$1.

The pseudocode given in the paper makes the data flow explicit. Around the last localization, the system retrieves recent map frames, segments the current image, matches those segments against map frames, computes graph distances to the target node, and selects the visible segment with minimum sub-goal cost. It then estimates depth, computes a traversability mask, projects into bird’s-eye view, and either follows the locally planned waypoints or falls back to RoboHop yaw control.

This modularity is central to the method’s open-set framing. The paper attributes deployability to the use of foundational models rather than to domain-specific training of a navigation policy.

5. Experimental evaluation and ablations

The main simulation evaluation uses HM3D, specifically the InstanceImageNav validation set, with $$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$2 episodes. Success is defined as being within $$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$3 m of the goal in at most $$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$4 steps. Results are reported for three path-length regimes: easy ($$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$5–$$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$6 m), hard ($$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$7–$$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$8 m), and full ($$C(\mathrm{edge}_{ij})=\alpha \cdot d_{ij}+\beta \cdot (1-T_{ij}),$$9–$d_{ij}$0 m) (Podgorski et al., 10 Sep 2025).

Protocol	Easy / Hard / Full	RoboHop / PixNav / TANGO
GT-Metric sub-goals	93.1 / 78.4 / 42.2 ; 65.7 / 44.1 / 15.7 ; 94.1 / 90.2 / 48.0	93.1/65.7/94.1 ; 78.4/44.1/90.2 ; 42.2/15.7/48.0
GT-Topological	78.4 / 58.8 / 25.5 ; 60.8 / 44.1 / 15.7 ; 74.5 / 65.7 / 30.4	78.4/60.8/74.5 ; 58.8/44.1/65.7 ; 25.5/15.7/30.4
No-GT (full pipeline)	43.6 / 34.6 / 13.7 ; 52.0 / 39.2 / 14.0 ; 61.8 / 43.1 / 21.6	43.6/52.0/61.8 ; 34.6/39.2/43.1 ; 13.7/14.0/21.6

Under GT-Metric sub-goals, TANGO exceeds RoboHop on hard and full trajectories, achieving $d_{ij}$1 and $d_{ij}$2, respectively. Under GT-Topological supervision, TANGO achieves $d_{ij}$3 on hard and $d_{ij}$4 on full, both above RoboHop and PixNav. In the full No-GT setting, TANGO reaches $d_{ij}$5, $d_{ij}$6, and $d_{ij}$7, outperforming both RoboHop and PixNav in all three regimes.

Ablations on the hard ($d_{ij}$8–$d_{ij}$9 m) regime isolate perception error sources. With oracle depth and traversability together with oracle segmentation, TANGO achieves $T_{ij}$0. Replacing oracle segmentation by SAM and Glue reduces performance to $T_{ij}$1. Using full SAM and Glue with Depth-Anything and SAM-traversability yields $T_{ij}$2. The paper summarizes this as segmentation and matching errors costing approximately $T_{ij}$3 points, while depth and traversability errors cost approximately $T_{ij}$4 points.

Ablation or variant	Result	Note
oracle depth+trav + oracle segmentation	65.7%	hard, $T_{ij}$5–$T_{ij}$6 m
oracle depth+trav + SAM+Glue segmentation	48.0%	hard, $T_{ij}$7–$T_{ij}$8 m
full SAM+Glue + Depth-Anything+SAM-trav	43.1%	hard, $T_{ij}$9–$\alpha,\beta \ge 0$0 m
without fallback / with fallback	62.1% / 73.8%	GT-Metric, hard, 250 steps

The auto-switching mechanism provides a substantial gain. In the GT-Metric, hard, $\alpha,\beta \ge 0$1-step setting, success rises from $\alpha,\beta \ge 0$2 without fallback to $\alpha,\beta \ge 0$3 with fallback, reported as $\alpha,\beta \ge 0$4 points. The paper also evaluates “seen but unvisited” goals, where the target object was observed during mapping but never reached. In that setting, TANGO achieves approximately $\alpha,\beta \ge 0$5 on hard and approximately $\alpha,\beta \ge 0$6 on full trajectories, described as comparable to original goals.

6. Interpretation, scope, and technical significance

TANGO is positioned as an alternative to two established visual-navigation design choices: globally-consistent 3D maps and learned controllers. Its contribution is not to eliminate structure, but to relocate it. Global structure is maintained in an object-level graph, while local safety and motion are derived online from monocular depth and traversability. This suggests a navigation regime in which long-horizon planning is handled topologically and short-horizon execution is handled metrically.

A common simplification is to treat the method as purely topological because its global sub-goals are object segments. The full system is more specific than that description. The topological layer alone is not the deployed controller; the distinguishing behavior comes from coupling that layer to continuous local trajectory prediction in BEV, together with automatic reversion to RoboHop when the metric layer fails. Conversely, it is also not a conventional map-based metric planner, because it does not require a 3D map and does not depend on a pre-trained navigation controller.

The ablation results clarify the present bottleneck. The dominant degradation arises from segmentation and matching rather than from monocular depth and traversability alone. This locates the main error surface in object-level perception and cross-frame association. The reported “seen but unvisited” performance further indicates that the graph abstraction can support goals that were visually observed during mapping but not explicitly traversed.

The paper characterizes the system as robust and deployable, reports both simulation and real-world tests, and states that the source code is publicly available. Within the paper’s framing, TANGO is an object-level topometric navigation pipeline that uses foundational models to preserve zero-shot, open-set applicability while improving over RoboHop and PixNav in HM3D evaluation (Podgorski et al., 10 Sep 2025).