---
title: 'R2R-Goal: Visual Navigation Instruction Benchmark'
url: https://www.emergentmind.com/topics/r2r-goal-dataset
type: topic
---

# R2R-Goal: Visual Navigation Instruction Benchmark

Searching arXiv for the cited papers to ground the article in current arXiv records.
R2R-Goal is the benchmark dataset introduced for **Goal-Conditioned Visual Navigation Instruction Generation (GoViG)**. It is designed for a task that inverts the conventional Room-to-Room paradigm: rather than following a natural-language instruction to produce actions, a system must generate a navigation instruction from egocentric visual observations of the initial state and the goal state alone [2508.09547]. The dataset combines simulator-derived trajectories and a smaller real-world subset, and it is explicitly positioned as a goal-conditioned, first-person, map-free, coordinate-free instruction-generation benchmark. In the surrounding literature, the name also requires disambiguation: it is distinct from the original Room-to-Room (R2R) instruction-following benchmark [1711.07280], from the destination-oriented R2R-Last benchmark discussed in later VLN work [2302.09230], and from the R2R-derived image knowledge base **R2R\_GP** introduced in BTK, which is not a benchmark named “R2R-Goal” [2603.26859].

## 1. Definition and task setting

R2R-Goal is defined around the GoViG task: given egocentric visual observations of the route start and the goal, the model must produce a natural-language navigation instruction [2508.09547]. The formal input consists of an initial visual sequence and a goal observation,
\[
\mathcal{O}=\{o_1,o_2,\dots,o_n\}, \quad o_g,
\]
where \(o_i, o_g \in \mathbb{R}^{H \times W \times 3}\), and the objective is to generate an instruction \(I\). The paper further states that each trajectory in R2R-Goal retains **an initial sequence of six egocentric observations and a final goal observation**.

This specification marks a deliberate departure from standard VLN formulations. In the original R2R benchmark, the input is a free-form route instruction and the output is a navigation trajectory through a Matterport3D viewpoint graph [1711.07280]. In R2R-Goal, by contrast, the input is visual and the output is linguistic. The benchmark therefore studies instruction generation from raw first-person RGB observations rather than instruction following. The paper frames this as addressing a gap left by prior methods that rely on structured inputs such as semantic maps, landmark annotations, panoramic observations, action traces, route coordinates, orientation signals, or GPS/map structure [2508.09547].

The benchmark also supports an auxiliary paired task, termed **Navigation Visualization** in the paper. Given a partial visual sequence \(\mathcal{O}_V=\{o_1,o_2,\dots,o_k\}\) and the goal observation \(o_g\), the model predicts the next visual observation \(o_{k+1}\). A second subtask, **Instruction Generation with Visual Cues**, uses \(\mathcal{O}_I=\{o_1,o_2,\dots,o_m\}\), the goal observation \(o_g\), and optionally \(I_{\text{prev}}\), to generate instruction \(I\). This dual structure makes R2R-Goal both a language-generation benchmark and a visual-forecasting benchmark [2508.09547].

## 2. Dataset construction and composition

R2R-Goal is constructed from both synthetic and real-world sources. The simulator-based portion is built from **R2R-CE** and **HA-R2R**, while the real-world subset is built from **GO Stanford**, **ReCon**, and **HuRoN** [2508.09547]. For the synthetic portion, the paper states that the dataset is constructed within the **HA-VLN simulation environment**, using path start and goal positions from HA-R2R and R2R-CE. An **A\***-based heuristic search identifies the shortest feasible navigation path, and the appendix further states that **dynamic re-planning** is triggered in real time upon unexpected obstacles. An egocentric camera mounted on the simulated agent continuously captures observations along each traversed path.

A central part of the construction process is scene-level segmentation and alignment using **Qwen2.5-VL-7B-Instruct**. The paper describes a two-stage procedure. First, navigation instructions are segmented into spatially coherent scenes, with the constraints that each segment corresponds to a navigable space and that all text is uniquely assigned. Post-processing then merges consecutive identical scenes and guarantees complete coverage. Second, the model analyzes the full visual sequence to detect scene transitions based on visual cues and instruction alignment, after which post-processing adjusts boundaries and removes gaps and overlaps. The result is a scene-level instruction generation dataset rather than a raw collection of trajectories [2508.09547].

The synthetic portion has official split counts, while the real-world subset is reserved for cross-domain testing.

| Split or subset | Count | Source |
|---|---:|---|
| Training | 48,490 | Synthetic |
| validation_seen | 3,573 | Synthetic |
| validation_unseen | 8,361 | Synthetic |
| test | 14,313 | Synthetic |
| Real-world subset | 150 trajectories | GO Stanford, ReCon, HuRoN |

The paper states that the simulator-based portion contains **74,737** total synthetic trajectories, and that the real-world subset contains **150 trajectories** that are **manually annotated** with corresponding natural-language navigation instructions [2508.09547]. It also states that trajectory lengths range from **8 to 29 steps**. The paper claims that the dataset covers “diverse indoor and outdoor environments,” but it does not list the exact top-20 category names or counts in the provided text. Likewise, it does not provide explicit values for average trajectory length, vocabulary size, average instruction length, number of unique scenes or buildings, or per-split instruction counts.

## 3. Data modalities and formal representation

At the instance level, each R2R-Goal example provides an initial egocentric visual sequence, a goal observation, and a corresponding navigation instruction [2508.09547]. The image modality is explicitly first-person RGB rather than panoramic, map-based, or symbolic. The paper emphasizes that the benchmark is **coordinate-free / action-free / map-free**, and that the goal is represented through a **goal observation image** rather than symbolic goal metadata.

The task formulations used with the dataset are given in exact notation. For the visualization subtask,
\[
\mathcal{O}_V=\{o_1,o_2,\dots,o_k\}, \; o_g \;\rightarrow\; o_{k+1}.
\]
For instruction generation with visual cues,
\[
\mathcal{O}_I=\{o_1,o_2,\dots,o_m\}, \; o_g,\; I_{\text{prev} \;\rightarrow\; I.
\]
The extracted text notes that the source PDF contains formatting problems in several equations, so the formulas are best treated as reproductions of the paper’s notation rather than corrected derivations.

The dataset is explicitly designed to support two model-side reasoning procedures introduced with GoViG. In **One-Pass reasoning**, the model predicts a future visual trajectory until a goal-similarity threshold is met, then generates the final instruction from sampled frames:
\[
I = F_\Theta\bigl(\{o_1, \hat{o}_{i_1}, \dots, \hat{o}_{i_{m-1}, o_g\}\bigr).
\]
In **Interleaved reasoning**, the model alternates between predicting the next frame and refining the instruction:
\[
I_t = F_\Theta\bigl(\{\,o_t, \dots, o_k, \hat{o}_{k+1}, \dots, \hat{o}_{k+t}, o_g, I_{t-1}\}\bigr).
\]
In both cases, the stopping condition is
\[
\mathrm{SSIM}(\hat{o}_{k+t}, o_g) > \tau
\]
with \(\tau = 0.7\) [2508.09547]. These are not properties of the dataset itself, but they are the benchmark usage formulations employed in experiments.

The paper also states that each navigation trajectory generates two distinct types of training instances: visualization instances, where \(k\) preceding frames plus the goal frame predict the next frame, and instruction instances, where initial, goal, and up to \(m-1\) intermediate frames predict the full trajectory instruction. This suggests a paired multimodal supervision regime in which temporal visual forecasting and language generation are jointly supported by the same underlying trajectory resource.

## 4. Evaluation protocol and empirical behavior

R2R-Goal is evaluated with standard text-generation metrics for the main task and visual-similarity metrics for the auxiliary visualization task [2508.09547]. For instruction generation, the paper reports **BLEU-4**, **CIDEr**, **METEOR**, and **ROUGE-L**. For navigation visualization, it reports **SSIM**, **PSNR**, **LPIPS**, and **DreamSim**. The real-world subset is used for **zero-shot generalization**, with **no additional fine-tuning**.

The benchmark compares against a mixed set of navigation-language and general multimodal baselines: **Speaker-Follower**, **LANA**, **GPT-4o**, **GPT-4o + CoT**, **C-Instructor**, **Anole-7B (Direct)**, **Anole-7B + CoT**, **Anole-7B + One-pass (Ours)**, and **Anole-7B + Interleaved (Ours)** [2508.09547]. The paper notes that several prior methods were retrained or adapted to accept only egocentric observations so that the comparison matches the R2R-Goal setting.

On the synthetic benchmark, the strongest reported instruction-generation numbers come from the interleaved reasoning variant. On **validation_unseen**, **Anole-7B + Interleaved (Ours)** reports **BLEU-4 0.32**, **CIDEr 0.20**, **METEOR 0.18**, and **ROUGE-L 0.21**. On **test**, the same method reports **BLEU-4 0.33**, **CIDEr 0.18**, **METEOR 0.20**, and **ROUGE-L 0.22** [2508.09547]. The abstract highlights **BLEU-4 = 0.33** and **CIDEr = 0.20** on the unseen validation set, while the table gives **0.32 / 0.20** for validation_unseen and **0.33 / 0.18** for test; the paper text itself therefore contains a minor inconsistency.

For the visualization benchmark on **val_unseen**, the paper reports that its method achieves **SSIM 0.69**, **PSNR 20.02**, **LPIPS 0.27**, and **DreamSim 0.13**, compared with **GPT-4o + DALL·E** at **0.29 / 9.57 / 0.72 / 0.61** and **Anole-7B (Direct)** at **0.50 / 14.98 / 0.39 / 0.27** [2508.09547]. This indicates that the auxiliary forecasting problem is nontrivial and quantitatively sensitive to modeling choices.

The real-world subset further functions as a transfer benchmark. Under zero-shot evaluation, **Anole-7B + Interleave** reports **BLEU-4 0.27**, **CIDEr 0.15**, **METEOR 0.19**, and **ROUGE-L 0.18** [2508.09547]. A plausible implication is that the dataset is designed not merely for simulator performance, but for studying whether first-person goal-conditioned instruction generation transfers across domains.

## 5. Relation to R2R and common terminological confusions

R2R-Goal is best understood relative to the original **R2R** benchmark. R2R, introduced with the Matterport3D Simulator, is an instruction-following dataset in which an embodied agent receives a natural-language route instruction, a starting pose, and first-person visual observations, and must navigate to the intended destination [1711.07280]. It contains **7,189 paths** and **21,567 navigation instructions**, split by scene into train, val seen, val unseen, and test, and it evaluates success by whether the final position lies within **3 m** of the goal. The goal in R2R is therefore already a target viewpoint, but it is specified implicitly through language rather than through a separate goal representation.

R2R-Goal reverses that interface. The benchmark input is visual evidence of start and goal states, while the output is the instruction text itself [2508.09547]. This makes it a generation benchmark rather than a navigation-control benchmark. The distinction matters because many later R2R-derived resources are easy to conflate despite addressing different problems.

One frequent confusion arises with **R2R-Last**, discussed in VLN-Trans. That setting uses only the last sentence of the original R2R instruction to describe the final destination, making it more destination-oriented than standard R2R, but it is still an instruction-following benchmark and not a dataset named R2R-Goal [2302.09230]. Another confusion arises with **R2R\_GP** in BTK. That resource is an **image knowledge base** derived from R2R instructions by extracting goal-related phrases with **Qwen3-4B** and generating semantically relevant images with **Flux-Schnell**; the paper explicitly states that it does **not** introduce a dataset literally named “R2R-Goal” [2603.26859]. R2R\_GP is therefore an auxiliary multimodal resource for navigation, not a standalone instruction-generation benchmark.

A further distinction is with **R2R-UNO**, which modifies R2R to introduce unexpected obstructions by changing both navigation graphs and visual observations [2407.21452]. That work remains instruction-conditioned VLN rather than goal-conditioned instruction generation. Taken together, these comparisons show that “R2R-Goal” denotes a specific benchmark associated with GoViG, not a generic label for any R2R-derived goal-related resource.

## 6. Limitations, release status, and research use

The paper presents R2R-Goal as a dataset intended for public use. It provides a code repository at **https://github.com/F1y1113/GoViG** and states in its reproducibility checklist that the novel dataset will be **made publicly available upon publication** with a license allowing **free usage for research purposes** [2508.09547]. The benchmark protocol includes official split definitions for the synthetic portion, evaluation metrics, and appendix-level implementation notes and prompts.

Several limitations are explicit in the provided description. First, the dataset is dominated by the synthetic portion: **74,737** simulator-derived trajectories versus a **150-trajectory** real-world subset [2508.09547]. Second, the synthetic portion relies on inherited instructions from R2R-CE and HA-R2R together with automatic scene segmentation and alignment, and the provided text does not report a quantitative segmentation/alignment accuracy measure. Third, the real-world annotation procedure is only partially specified: the paper states that **150 trajectories** were manually annotated, but does not provide annotator counts, agreement statistics, multiple-reference protocols, or a detailed quality-assurance procedure.

The paper also omits several statistics that would ordinarily be central to a dataset specification, including vocabulary size, average instruction length, average trajectory length, number of unique environments or scans, and per-split instruction counts [2508.09547]. This suggests that, although the benchmark is well specified at the task and split level, its descriptive corpus documentation is less complete than its evaluation protocol.

Within embodied AI research, R2R-Goal occupies a distinct methodological niche. It benchmarks whether a model can synthesize route language from sparse first-person visual evidence, without explicit maps, coordinates, action labels, or panoramic state descriptions. This suggests a research role complementary to instruction-following VLN: rather than optimizing action prediction conditioned on language, R2R-Goal tests whether latent spatial understanding can be externalized as navigation instruction text from goal-conditioned egocentric observations alone.

Source: https://www.emergentmind.com/topics/r2r-goal-dataset