R2R-VP: Visual Prompt Navigation Benchmark

• R2R-VP is a discrete visual-prompt navigation benchmark that replaces natural-language instructions with top-view maps showing rendered trajectory prompts.
• It employs a graph-based approach using panoramic observations and partial graph construction to guide an agent's node selection in complex indoor environments.
• VPNet, built on DUET with ViT-based encoding and cross-modal reasoning, demonstrates that clear visual prompts significantly improve route guidance and data efficiency.

R2R-VP denotes Room-to-Room with Visual Prompts, a discrete-environment benchmark for Visual Prompt Navigation (VPN) that transfers the original R2R task from language-conditioned route following to visual-prompt-guided route following. In this formulation, the natural-language instruction is removed and replaced by a 2D top-view map containing a rendered trajectory prompt, while the underlying Matterport3D viewpoint-graph navigation problem, panoramic observations, and goal-reaching objective are retained. The benchmark therefore studies whether route intent can be conveyed through a visually marked path rather than through text, and it serves as the discrete counterpart to R2R-CE-VP in continuous environments (Feng et al., 3 Aug 2025).

1. Definition and nomenclature

R2R-VP is built directly from Room-to-Room (R2R) episodes. In the original R2R setting, an agent is given a start viewpoint, a natural-language route instruction, and must navigate over a scene’s viewpoint graph to the target. In R2R-VP, the episode structure is preserved, but the language instruction is replaced by a top-down map image containing a rendered visual trajectory prompt. The resulting guidance is visual, spatially grounded, and map-based rather than symbolic and linguistic (Feng et al., 3 Aug 2025).

The benchmark is explicitly discrete. The paper separately defines R2R-CE-VP as the continuous-environment counterpart derived from R2R-CE, but R2R-VP itself remains a graph-navigation task with node selection rather than low-level motor control (Feng et al., 3 Aug 2025).

The term is also subject to a genuine nomenclatural ambiguity in the VLN literature. A later graph-based VLN-CE paper states that “R2R-VP usually refers to this classic setting in practice”, namely navigation over panoramic viewpoints / graph nodes with viewpoint transitions or panoramic decisions. In contrast, the VPN paper introduces R2R-VP as Room-to-Room with Visual Prompts. Accordingly, in recent work the same string can denote either the classic viewpoint-graph regime in a loose practical sense or the specific visual-prompt benchmark introduced for VPN (Ye et al., 24 Dec 2025).

2. Formal problem setting

R2R-VP is formulated as navigation on an undirected graph

$\mathcal{G} = (\mathcal{V}, \mathcal{E}),$

where $\mathcal{V}$ denotes navigable nodes and $\mathcal{E}$ denotes connectivity edges. Starting from an initial node and guided by visual prompts, the agent explores $\mathcal{G}$ and attempts to reach the target node (Feng et al., 3 Aug 2025).

The agent’s inputs comprise three principal components. First, it receives the visual prompt map or maps, encoded after processing as prompt embeddings. Second, at step $t$ it observes panoramic views

$V_t=\{v_{t,i}\}_{i=1}^{n}.$

Third, it identifies a navigable subset

$\mathcal{N}(V_t)\subseteq V_t,$

whose elements correspond to neighboring nodes and their spatial coordinates. The paper further specifies relative-orientation information through a direction embedding $W_{ij}^d$ and navigability-type information through a navigability embedding $W_{ij}^n$ (Feng et al., 3 Aug 2025).

The action space is high-level and graph-based. Candidate actions are represented as

$$\hat{v}=(\text{stop}, nav_1, nav_2, \dots, nav_{v'}),$$

and the model scores them by

$\mathcal{V}$0

After masking the current node and revisits, the agent selects the highest-scoring candidate and then navigates to it via the shortest path in the topological graph $\mathcal{V}$1. This is a node-selection regime rather than a continuous-control regime (Feng et al., 3 Aug 2025).

A central structural property is that the agent does not know the full graph in advance. Instead, it incrementally builds a partial graph

$\mathcal{V}$2

whose node types include the current node, visited nodes, and navigable nodes (Feng et al., 3 Aug 2025).

3. Dataset construction and prompt representation

The R2R-VP dataset is produced by a four-step transfer pipeline from R2R. First, for each episode, a 2D top-view map is generated from the 90 MP3D environments, “following a procedure similar to LED.” Second, the pixel coordinates of all viewpoints are computed on the top-view map and connected sequentially with arrows to represent the trajectory. Third, a square crop centered on the trajectory is extracted, with side length equal to the greater of trajectory width or height plus a 60-pixel margin. Fourth, black borders are removed by detecting the bounding box of all non-zero pixels (Feng et al., 3 Aug 2025).

After preprocessing, all prompt maps are resized to $\mathcal{V}$3 before encoding. The prompt itself is intended to mark the trajectory path, waypoints / key points, and the direction of traversal using arrows. The paper does not provide explicit 3D-to-2D projection formulas or a formal world-to-pixel transformation equation, so the geometric projection procedure is operationally described rather than analytically specified (Feng et al., 3 Aug 2025).

Multi-floor episodes are handled through Order-Aware Floor Concatenation (OAFC). For the $\mathcal{V}$4-th prompt map,

$\mathcal{V}$5

and the sequence is concatenated as

$\mathcal{V}$6

where $\mathcal{V}$7 is an order embedding that preserves floor traversal order. This mechanism is motivated by the multi-floor structure of MP3D scenes; among the 90 MP3D scenes, 32 have 1 floor, 32 have 2 floors, 17 have 3 floors, 8 have 4 floors, and 1 has 5 floors (Feng et al., 3 Aug 2025).

The discrete R2R-VP training set contains 61 scenes and 4,638 episodes. For comparison, the original discrete R2R training split had 61 scenes and 4,675 episodes, implying that some episodes were removed during transfer. The paper also constructs visual-prompt training data from PREVALENT, yielding 60 scenes and 177,134 episodes, and from ScaleVLN, yielding 523 scenes and 1,600,945 episodes after retaining HM3D scenes or floors with acceptable top-view map quality (Feng et al., 3 Aug 2025).

4. VPNet architecture and training protocol

The benchmark paper introduces VPNet as the dedicated baseline model for VPN. In the discrete R2R-VP setting, VPNet is based on DUET and comprises a ViT-based Visual Prompt Encoder, a Node Embedding module, and a Graph-aware Cross-modal Encoder; the implementation details also state that a local cross-modal encoder is used in the discrete setting (Feng et al., 3 Aug 2025).

Prompt maps are encoded with ViT-B/16. Panoramic scene views are likewise represented with visual features $\mathcal{V}$8, which are combined with direction and navigability signals:

$\mathcal{V}$9

A two-layer panoramic self-attention encoder then contextualizes the views:

$\mathcal{E}$0

The node embedding at time $\mathcal{E}$1 is formed by adding step embedding, position embedding, and the averaged contextualized view features:

$\mathcal{E}$2

A special stop node is appended, and prompt embeddings and node embeddings are fused through cross-modal and graph-aware reasoning before the policy head scores actions by $\mathcal{E}$3 (Feng et al., 3 Aug 2025).

Training uses a mixture of behavior cloning and DAgger, with the shortest-path next node on the on-policy partial graph serving as the pseudo-demonstrator action in DAgger. For R2R-VP, the loss mixing coefficient is $\mathcal{E}$4. The reported implementation settings are 400k iterations, single NVIDIA RTX A5000 GPU, batch size 10, and learning rate $\mathcal{E}$5. Architectural sizes are also specified: 12 transformer layers for the visual prompt encoder, 2 layers for the panoramic view encoder, 4 layers for the graph-aware cross-modal encoder, 4 layers for the local cross-modal encoder, and hidden size 768 (Feng et al., 3 Aug 2025).

Two augmentation families are central to the reported performance. Prompt-view augmentation rotates the 2D top-view prompt map by one of

$\mathcal{E}$6

while agent-view augmentation randomly samples the initial heading from $\mathcal{E}$7. The paper emphasizes that, unlike in VLN, the initial heading in VPN is independent of the prompt, making such augmentation particularly suitable. At the trajectory level, training is further expanded with PREVALENT-derived and ScaleVLN-derived VPN episodes (Feng et al., 3 Aug 2025).

5. Prompt design, quantitative results, and ablations

The paper evaluates several prompt forms and concludes that trajectory-centered cropping and explicit trajectory rendering are decisive. A full-scene top-view map without trajectory-centered cropping performs poorly because many episodes from the same scene share the same map image while differing only in route. A cropped top-view map without the visual trajectory already works “surprisingly well,” which the authors relate to rough destination-area information, but the best-performing prompt form is “lines only”: a clean trajectory rendering without text labels. The paper’s interpretation is that lines already provide sufficient spatiotemporal guidance, whereas text may obscure details around waypoints (Feng et al., 3 Aug 2025).

Quantitatively, the prompt-type ablation on Val Unseen reports the following progression: full-map prompt NE 7.07, SR 33.94, SPL 29.36; cropped-map-only prompt NE 5.51, SR 45.83, SPL 36.78; a stronger cropped-map variant NE 5.13, SR 50.18, SPL 40.54; lines + text prompt NE 3.74, SR 65.36, SPL 54.81; and lines only prompt NE 2.18, SR 65.92, SPL 56.17. On Val Seen, the lines only prompt reaches NE 0.05, SR 100.00, SPL 99.77 (Feng et al., 3 Aug 2025).

The main benchmark results show three training regimes. With R2R only, VPNet obtains Val Seen: NE 3.37, OSR 72.41, SR 66.69, SPL 62.13; Val Unseen: NE 5.43, OSR 59.53, SR 51.23, SPL 43.47; Test Unseen: NE 5.11, OSR 62.33, SR 52.40, SPL 42.87. With R2R + PRE, the numbers improve to Val Seen: NE 0.05, OSR 100, SR 100, SPL 99.77; Val Unseen: NE 2.18, OSR 76.76, SR 65.92, SPL 56.17; Test Unseen: NE 1.94, OSR 78.21, SR 66.38, SPL 56.26. With R2R + PRE + SCA, the reported results are Val Seen: NE 0.14, OSR 99.41, SR 99.41, SPL 99.08; Val Unseen: NE 0.48, OSR 97.45, SR 96.68, SPL 94.84; Test Unseen: NE 0.31, OSR 98.56, SR 97.56, SPL 94.60 (Feng et al., 3 Aug 2025).

The augmentation ablation identifies the best view-level configuration as both prompt-view and agent-view augmentation, yielding TL 9.92, NE 0.48, OSR 97.45, SR 96.68, SPL 94.84. Prompt-view augmentation alone is also strong, with NE 0.62, SR 96.41, SPL 94.37, and the paper notes that prompt rotation provides larger gains than randomizing the initial heading (Feng et al., 3 Aug 2025).

Robustness tests inject 20% salt-and-pepper noise into the prompt or remove the first-step visual cue. Relative to the clean-prompt result TL 9.92, NE 0.48, OSR 97.45, SR 96.68, SPL 94.84, salt-and-pepper noise yields TL 11.73, NE 1.24, OSR 93.53, SR 90.34, SPL 85.40, while removing the first-step cue yields TL 12.48, NE 2.13, OSR 89.31, SR 85.97, SPL 74.98. The larger degradation from removing the first cue indicates that early trajectory information is especially important (Feng et al., 3 Aug 2025).

The paper also frames the benchmark in relation to standard VLN baselines. It states that VPNet in VPN outperforms DUET in VLN while using only 1,600,945 ScaleVLN trajectories, compared to DUET’s 4,941,710, presenting this as evidence of data efficiency in the VPN framework (Feng et al., 3 Aug 2025).

6. Significance, adjacent research, and limitations

R2R-VP is positioned as a hybrid between VLN and ImageNav. Like VLN, it can encode an intended route, not merely a destination. Like ImageNav, it presents information in a direct visual form rather than in language. The main empirical lesson drawn by the paper is that visual prompts can successfully replace language in R2R-style navigation, provided that the prompt clearly encodes route geometry and traversal direction (Feng et al., 3 Aug 2025).

Several specific findings follow from the reported ablations. First, route structure matters more than text labels: the best prompt is a clean trajectory rendering rather than a heavier symbolic overlay. Second, cropping and prompt-specific map design are crucial: if the map is too broad, episode-specific route identity becomes weaker. Third, prompt diversity matters: orientation-invariant learning from rotated prompts substantially improves generalization. Fourth, scaling trajectory data is highly effective: PREVALENT and ScaleVLN augmentation drive the largest gains, to the point that, as the paper states, with large-scale augmented training, R2R-VP becomes nearly solved under this benchmark (Feng et al., 3 Aug 2025).

The benchmark also has explicit constraints. It depends on the availability and quality of a usable 2D top-view map, and some scenes or floors are excluded because the top-view reconstructions are poor. Prompt generation in the benchmark is based on ground-truth trajectories, whereas real human-drawn prompts may be noisier or less precise. The paper further notes that formal world-to-map projection equations are under-specified, which may hinder exact replication without the released code (Feng et al., 3 Aug 2025).

In adjacent VLN research, graph-based continuous-environment methods remain conceptually close to R2R-VP even when their evaluation benchmark differs. ETP-R1, for example, is evaluated on R2R-CE and RxR-CE, not on the original discrete benchmark, but its high-level planner selects graph nodes / waypoints / topological subgoals while a low-level controller handles execution. The paper explicitly argues that this planning interface is very close to the viewpoint/node-level decision process that R2R-VP researchers care about. A plausible implication is that the topological abstractions emphasized by R2R-VP remain central even as VLN systems move toward continuous embodied execution and online reinforcement fine-tuning (Ye et al., 24 Dec 2025).