R2R-Goal: Visual Navigation Instruction Benchmark

• The paper introduces the R2R-Goal benchmark, which inverts the typical Room-to-Room paradigm by generating natural-language instructions from egocentric start and goal images.
• It employs a dual-task framework that integrates visual forecasting and language generation through both one-pass and interleaved reasoning approaches with metrics like BLEU-4 and CIDEr.
• The dataset combines extensive synthetic trajectories with a small, manually annotated real-world subset to facilitate zero-shot cross-domain evaluation.

Searching arXiv for the cited papers to ground the article in current arXiv records. R2R-Goal is the benchmark dataset introduced for Goal-Conditioned Visual Navigation Instruction Generation (GoViG). It is designed for a task that inverts the conventional Room-to-Room paradigm: rather than following a natural-language instruction to produce actions, a system must generate a navigation instruction from egocentric visual observations of the initial state and the goal state alone (Wu et al., 13 Aug 2025). The dataset combines simulator-derived trajectories and a smaller real-world subset, and it is explicitly positioned as a goal-conditioned, first-person, map-free, coordinate-free instruction-generation benchmark. In the surrounding literature, the name also requires disambiguation: it is distinct from the original Room-to-Room (R2R) instruction-following benchmark (Anderson et al., 2017), from the destination-oriented R2R-Last benchmark discussed in later VLN work (Zhang et al., 2023), and from the R2R-derived image knowledge base R2R_GP introduced in BTK, which is not a benchmark named “R2R-Goal” (Yang et al., 27 Mar 2026).

1. Definition and task setting

R2R-Goal is defined around the GoViG task: given egocentric visual observations of the route start and the goal, the model must produce a natural-language navigation instruction (Wu et al., 13 Aug 2025). The formal input consists of an initial visual sequence and a goal observation,

$\mathcal{O}=\{o_1,o_2,\dots,o_n\}, \quad o_g,$

where $o_i, o_g \in \mathbb{R}^{H \times W \times 3}$, and the objective is to generate an instruction $I$. The paper further states that each trajectory in R2R-Goal retains an initial sequence of six egocentric observations and a final goal observation.

This specification marks a deliberate departure from standard VLN formulations. In the original R2R benchmark, the input is a free-form route instruction and the output is a navigation trajectory through a Matterport3D viewpoint graph (Anderson et al., 2017). In R2R-Goal, by contrast, the input is visual and the output is linguistic. The benchmark therefore studies instruction generation from raw first-person RGB observations rather than instruction following. The paper frames this as addressing a gap left by prior methods that rely on structured inputs such as semantic maps, landmark annotations, panoramic observations, action traces, route coordinates, orientation signals, or GPS/map structure (Wu et al., 13 Aug 2025).

The benchmark also supports an auxiliary paired task, termed Navigation Visualization in the paper. Given a partial visual sequence $\mathcal{O}_V=\{o_1,o_2,\dots,o_k\}$ and the goal observation $o_g$, the model predicts the next visual observation $o_{k+1}$. A second subtask, Instruction Generation with Visual Cues, uses $\mathcal{O}_I=\{o_1,o_2,\dots,o_m\}$, the goal observation $o_g$, and optionally $I_{\text{prev}}$, to generate instruction $I$. This dual structure makes R2R-Goal both a language-generation benchmark and a visual-forecasting benchmark (Wu et al., 13 Aug 2025).

2. Dataset construction and composition

R2R-Goal is constructed from both synthetic and real-world sources. The simulator-based portion is built from R2R-CE and HA-R2R, while the real-world subset is built from GO Stanford, ReCon, and HuRoN (Wu et al., 13 Aug 2025). For the synthetic portion, the paper states that the dataset is constructed within the HA-VLN simulation environment, using path start and goal positions from HA-R2R and R2R-CE. An A*-based heuristic search identifies the shortest feasible navigation path, and the appendix further states that dynamic re-planning is triggered in real time upon unexpected obstacles. An egocentric camera mounted on the simulated agent continuously captures observations along each traversed path.

A central part of the construction process is scene-level segmentation and alignment using Qwen2.5-VL-7B-Instruct. The paper describes a two-stage procedure. First, navigation instructions are segmented into spatially coherent scenes, with the constraints that each segment corresponds to a navigable space and that all text is uniquely assigned. Post-processing then merges consecutive identical scenes and guarantees complete coverage. Second, the model analyzes the full visual sequence to detect scene transitions based on visual cues and instruction alignment, after which post-processing adjusts boundaries and removes gaps and overlaps. The result is a scene-level instruction generation dataset rather than a raw collection of trajectories (Wu et al., 13 Aug 2025).

The synthetic portion has official split counts, while the real-world subset is reserved for cross-domain testing.

Split or subset	Count	Source
Training	48,490	Synthetic
validation_seen	3,573	Synthetic
validation_unseen	8,361	Synthetic
test	14,313	Synthetic
Real-world subset	150 trajectories	GO Stanford, ReCon, HuRoN

The paper states that the simulator-based portion contains 74,737 total synthetic trajectories, and that the real-world subset contains 150 trajectories that are manually annotated with corresponding natural-language navigation instructions (Wu et al., 13 Aug 2025). It also states that trajectory lengths range from 8 to 29 steps. The paper claims that the dataset covers “diverse indoor and outdoor environments,” but it does not list the exact top-20 category names or counts in the provided text. Likewise, it does not provide explicit values for average trajectory length, vocabulary size, average instruction length, number of unique scenes or buildings, or per-split instruction counts.

3. Data modalities and formal representation

At the instance level, each R2R-Goal example provides an initial egocentric visual sequence, a goal observation, and a corresponding navigation instruction (Wu et al., 13 Aug 2025). The image modality is explicitly first-person RGB rather than panoramic, map-based, or symbolic. The paper emphasizes that the benchmark is coordinate-free / action-free / map-free, and that the goal is represented through a goal observation image rather than symbolic goal metadata.

The task formulations used with the dataset are given in exact notation. For the visualization subtask,

$o_i, o_g \in \mathbb{R}^{H \times W \times 3}$0

For instruction generation with visual cues,

$o_i, o_g \in \mathbb{R}^{H \times W \times 3}$1

The extracted text notes that the source PDF contains formatting problems in several equations, so the formulas are best treated as reproductions of the paper’s notation rather than corrected derivations.

The dataset is explicitly designed to support two model-side reasoning procedures introduced with GoViG. In One-Pass reasoning, the model predicts a future visual trajectory until a goal-similarity threshold is met, then generates the final instruction from sampled frames: $o_i, o_g \in \mathbb{R}^{H \times W \times 3}$2 In Interleaved reasoning, the model alternates between predicting the next frame and refining the instruction: $o_i, o_g \in \mathbb{R}^{H \times W \times 3}$3 In both cases, the stopping condition is

$o_i, o_g \in \mathbb{R}^{H \times W \times 3}$4

with $o_i, o_g \in \mathbb{R}^{H \times W \times 3}$5 (Wu et al., 13 Aug 2025). These are not properties of the dataset itself, but they are the benchmark usage formulations employed in experiments.

The paper also states that each navigation trajectory generates two distinct types of training instances: visualization instances, where $o_i, o_g \in \mathbb{R}^{H \times W \times 3}$6 preceding frames plus the goal frame predict the next frame, and instruction instances, where initial, goal, and up to $o_i, o_g \in \mathbb{R}^{H \times W \times 3}$7 intermediate frames predict the full trajectory instruction. This suggests a paired multimodal supervision regime in which temporal visual forecasting and language generation are jointly supported by the same underlying trajectory resource.

4. Evaluation protocol and empirical behavior

R2R-Goal is evaluated with standard text-generation metrics for the main task and visual-similarity metrics for the auxiliary visualization task (Wu et al., 13 Aug 2025). For instruction generation, the paper reports BLEU-4, CIDEr, METEOR, and ROUGE-L. For navigation visualization, it reports SSIM, PSNR, LPIPS, and DreamSim. The real-world subset is used for zero-shot generalization, with no additional fine-tuning.

The benchmark compares against a mixed set of navigation-language and general multimodal baselines: Speaker-Follower, LANA, GPT-4o, GPT-4o + CoT, C-Instructor, Anole-7B (Direct), Anole-7B + CoT, Anole-7B + One-pass (Ours), and Anole-7B + Interleaved (Ours) (Wu et al., 13 Aug 2025). The paper notes that several prior methods were retrained or adapted to accept only egocentric observations so that the comparison matches the R2R-Goal setting.

On the synthetic benchmark, the strongest reported instruction-generation numbers come from the interleaved reasoning variant. On validation_unseen, Anole-7B + Interleaved (Ours) reports BLEU-4 0.32, CIDEr 0.20, METEOR 0.18, and ROUGE-L 0.21. On test, the same method reports BLEU-4 0.33, CIDEr 0.18, METEOR 0.20, and ROUGE-L 0.22 (Wu et al., 13 Aug 2025). The abstract highlights BLEU-4 = 0.33 and CIDEr = 0.20 on the unseen validation set, while the table gives 0.32 / 0.20 for validation_unseen and 0.33 / 0.18 for test; the paper text itself therefore contains a minor inconsistency.

For the visualization benchmark on val_unseen, the paper reports that its method achieves SSIM 0.69, PSNR 20.02, LPIPS 0.27, and DreamSim 0.13, compared with GPT-4o + DALL·E at 0.29 / 9.57 / 0.72 / 0.61 and Anole-7B (Direct) at 0.50 / 14.98 / 0.39 / 0.27 (Wu et al., 13 Aug 2025). This indicates that the auxiliary forecasting problem is nontrivial and quantitatively sensitive to modeling choices.

The real-world subset further functions as a transfer benchmark. Under zero-shot evaluation, Anole-7B + Interleave reports BLEU-4 0.27, CIDEr 0.15, METEOR 0.19, and ROUGE-L 0.18 (Wu et al., 13 Aug 2025). A plausible implication is that the dataset is designed not merely for simulator performance, but for studying whether first-person goal-conditioned instruction generation transfers across domains.

5. Relation to R2R and common terminological confusions

R2R-Goal is best understood relative to the original R2R benchmark. R2R, introduced with the Matterport3D Simulator, is an instruction-following dataset in which an embodied agent receives a natural-language route instruction, a starting pose, and first-person visual observations, and must navigate to the intended destination (Anderson et al., 2017). It contains 7,189 paths and 21,567 navigation instructions, split by scene into train, val seen, val unseen, and test, and it evaluates success by whether the final position lies within 3 m of the goal. The goal in R2R is therefore already a target viewpoint, but it is specified implicitly through language rather than through a separate goal representation.

R2R-Goal reverses that interface. The benchmark input is visual evidence of start and goal states, while the output is the instruction text itself (Wu et al., 13 Aug 2025). This makes it a generation benchmark rather than a navigation-control benchmark. The distinction matters because many later R2R-derived resources are easy to conflate despite addressing different problems.

One frequent confusion arises with R2R-Last, discussed in VLN-Trans. That setting uses only the last sentence of the original R2R instruction to describe the final destination, making it more destination-oriented than standard R2R, but it is still an instruction-following benchmark and not a dataset named R2R-Goal (Zhang et al., 2023). Another confusion arises with R2R_GP in BTK. That resource is an image knowledge base derived from R2R instructions by extracting goal-related phrases with Qwen3-4B and generating semantically relevant images with Flux-Schnell; the paper explicitly states that it does not introduce a dataset literally named “R2R-Goal” (Yang et al., 27 Mar 2026). R2R_GP is therefore an auxiliary multimodal resource for navigation, not a standalone instruction-generation benchmark.

A further distinction is with R2R-UNO, which modifies R2R to introduce unexpected obstructions by changing both navigation graphs and visual observations (Hong et al., 2024). That work remains instruction-conditioned VLN rather than goal-conditioned instruction generation. Taken together, these comparisons show that “R2R-Goal” denotes a specific benchmark associated with GoViG, not a generic label for any R2R-derived goal-related resource.

6. Limitations, release status, and research use

The paper presents R2R-Goal as a dataset intended for public use. It provides a code repository at https://github.com/F1y1113/GoViG and states in its reproducibility checklist that the novel dataset will be made publicly available upon publication with a license allowing free usage for research purposes (Wu et al., 13 Aug 2025). The benchmark protocol includes official split definitions for the synthetic portion, evaluation metrics, and appendix-level implementation notes and prompts.

Several limitations are explicit in the provided description. First, the dataset is dominated by the synthetic portion: 74,737 simulator-derived trajectories versus a 150-trajectory real-world subset (Wu et al., 13 Aug 2025). Second, the synthetic portion relies on inherited instructions from R2R-CE and HA-R2R together with automatic scene segmentation and alignment, and the provided text does not report a quantitative segmentation/alignment accuracy measure. Third, the real-world annotation procedure is only partially specified: the paper states that 150 trajectories were manually annotated, but does not provide annotator counts, agreement statistics, multiple-reference protocols, or a detailed quality-assurance procedure.

The paper also omits several statistics that would ordinarily be central to a dataset specification, including vocabulary size, average instruction length, average trajectory length, number of unique environments or scans, and per-split instruction counts (Wu et al., 13 Aug 2025). This suggests that, although the benchmark is well specified at the task and split level, its descriptive corpus documentation is less complete than its evaluation protocol.

Within embodied AI research, R2R-Goal occupies a distinct methodological niche. It benchmarks whether a model can synthesize route language from sparse first-person visual evidence, without explicit maps, coordinates, action labels, or panoramic state descriptions. This suggests a research role complementary to instruction-following VLN: rather than optimizing action prediction conditioned on language, R2R-Goal tests whether latent spatial understanding can be externalized as navigation instruction text from goal-conditioned egocentric observations alone.