NavigScene: Beyond-Visual-Range Driving Dataset

• NavigScene is an auxiliary dataset that integrates local sensor data with global navigation cues for autonomous driving.
• It operationalizes three paradigms—Navigation-guided Reasoning, Preference Optimization, and Vision-Language-Action—to enable anticipatory route planning.
• Empirical results show marked improvements in perception, planning, and generalization on QA and end-to-end driving benchmarks.

Searching arXiv for the specified NavigScene paper and closely related work to ground the article and citations. arxiv_search(query="(Peng et al., 7 Jul 2025) NavigScene Bridging Local Perception and Global Navigation for Beyond-Visual-Range Autonomous Driving", max_results=5) NavigScene is an auxiliary navigation-guided natural language dataset for autonomous driving that is designed to bridge local sensor perception and global, beyond-visual-range navigation context. It was introduced to address a specific deficiency in contemporary driving systems: strong performance in local perception, prediction, planning, and question answering does not imply robust use of broader route intent, such as the next maneuver, its distance, the type of intersection ahead, or the presence of traffic signals. NavigScene operationalizes this broader context by pairing multi-view driving scenes with natural-language navigation guidance and by defining three corresponding usage paradigms—Navigation-guided Reasoning, Navigation-guided Preference Optimization, and a Navigation-guided Vision-Language-Action model—for question answering and end-to-end driving tasks (Peng et al., 7 Jul 2025).

1. Conceptual role and motivation

NavigScene is framed around the distinction between local visual information and global navigation information. The motivating claim is that autonomous driving systems have advanced substantially in tasks grounded in nearby camera or LiDAR evidence, yet remain weak at incorporating the route-level cues that human drivers routinely obtain from map applications. This matters most in the beyond-visual-range (BVR) regime, where the next relevant maneuver may be approximately 150 m ahead or more, outside the effective sensing range of onboard cameras and LiDAR (Peng et al., 7 Jul 2025).

The dataset is therefore “auxiliary” in a precise sense: it is not presented as a standalone replacement for perception corpora, but as a post-training or conditioning resource for route-aware driving intelligence. It is derived from nuScenes and NAVSIM, and split into NavigScene-nuScenes and NavigScene-NAVSIM. Its core function is to expose models to future road structure before that structure becomes directly visible. The represented guidance includes the intended maneuver, the distance until that maneuver, the type of intersection or interchange, whether traffic signals are involved, and route information that may lie outside the sensor’s visual range.

A plausible implication is that NavigScene shifts the informational role of a driving model from predominantly reactive scene interpretation toward route-conditioned anticipatory reasoning. In the paper’s terminology, the objective is to make the system reason more like a human driver with access to Google Maps-style route instructions.

2. Dataset construction and annotation pipeline

NavigScene is built through a two-stage process: visual generation followed by text generation (Peng et al., 7 Jul 2025).

In the visual stage, source and destination coordinates are determined from the scene origin and 3D translation vectors. Translation in meters is converted into latitude and longitude using

$$\phi' = \phi + \frac{180}{\pi} \cdot \frac{\Delta y}{R}, \qquad \lambda' = \lambda + \frac{180}{\pi} \cdot \frac{\Delta x}{R \cdot \cos\!\left(\frac{\pi}{180} \cdot \phi\right)},$$

where $R$ is Earth’s radius, approximately 6,378,137 m. The $z$-axis is ignored because it does not affect horizontal navigation. The route stream is then synthesized with Google Maps APIs: the Direction API for routing, the Static Map API for sequential route images, and the Distance Matrix API for distance and duration estimation. The construction assumes constant velocity in order to synthesize a realistic navigation video aligned with the driving scene.

In the text stage, the route frames are converted into natural-language navigation instructions with a VLM, explicitly stated as GPT-4o. For each sampled navigation sequence, the pipeline generates $N$ candidate descriptions and selects the final one through a self-consistency procedure. The scoring function combines directional and intersection agreement, distance agreement, and lexical overlap:

$$S_{over}(a_i, a_j) = \eta_1 S_{inter}(a_i, a_j) + \eta_2 S_{dist}(a_i, a_j) + \eta_3 S_{word}(a_i, a_j),$$

with $\eta_1 > \eta_2 > \eta_3$. The final instruction is chosen by

$$a^* = \arg\max_{a_i \in A} \sum_{j \neq i} S_{over}(a_i, a_j).$$

This design encodes a clear priority hierarchy: directional correctness is treated as most important, distance precision next, and lexical similarity last. The resulting guidance is therefore not merely descriptive text, but a distilled route instruction intended to be consistent across multiple generated candidates.

3. Three usage paradigms

NavigScene is coupled to three modeling paradigms rather than being introduced as a passive annotation resource. These paradigms correspond to progressively tighter integration of navigation guidance into the model stack (Peng et al., 7 Jul 2025).

Paradigm	Core mechanism	Intended use
Navigation-guided Reasoning	Concatenate guidance with the question prompt	QA-style reasoning
Navigation-guided Preference Optimization	Extend DPO with navigation-aware summarized preferences	Post-training for robustness and generalization
Navigation-guided Vision-Language-Action	Fuse navigation-aware VLM features with conventional driving features	End-to-end driving

Navigation-guided Reasoning is the simplest formulation. It prepends navigation guidance to the QA prompt:

$a = M(v, g \oplus q),$

where $M$ is the VLM, $v$ is the multi-view image or video input, $R$0 is the navigation guidance, and $R$1 is the question. This is implemented as Navigation-guided Supervised Fine-Tuning (NSFT).

Navigation-guided Preference Optimization (NPO) extends Direct Preference Optimization by introducing an auxiliary summarization task biased toward navigation-relevant compression. Given an answer $R$2, the model generates a summary

$R$3

where pt requests a simple, concise summary that preserves important driving information. The summarized-answer reward is

$R$4

while the original-answer reward is

$R$5

The optimization objective is

$R$6

The paper further defines a mutual-information-based navigation relevance term and simplifies it to

$R$7

This makes NPO a preference-learning procedure explicitly biased toward shorter and more navigation-aligned summaries.

Navigation-guided Vision-Language-Action (NVLA) injects navigation-aware VLM features into a conventional driving model. The baseline task head is

$R$8

where $R$9 is the BEV encoder and $z$0 is the task head. The navigation-augmented form is

$z$1

Here $z$2 is a sparsity-reduction MLP for the high-dimensional VLM output, and $z$3 is a fusion MLP that merges VLM and BEV features.

4. Benchmarks, backbones, and training setup

The evaluation spans both QA and end-to-end driving benchmarks (Peng et al., 7 Jul 2025).

For QA, the paper uses DriveLM-nuScenes, with approximately 700 scenes, several hundred questions per scene, and six-camera images, of which 200 scenes are used for testing and the rest for training. It also uses NuInstruct, which contains over 10,000 pairs with six-camera inputs and evaluates perception, prediction, risk assessment, and planning subtasks.

For end-to-end driving, the paper uses nuScenes, consisting of 1,000 scenes of 20-second clips with six cameras, LiDAR, and radar, and NAVSIM, a closed-loop simulation benchmark based on nuPlan with 120 hours of data, eight cameras, and five LiDAR sensors.

Three VLM backbones are post-trained: Llama-Adapter-7B, LLaVA-v1.6-Mistral-7B, and Qwen2.5-VL-Instruct-7B. The reported post-training configuration uses LoRA rank = 16, learning rate $z$4, $z$5, and maximum output length 128 tokens. The end-to-end driving backbones are VAD and SparseDrive. Their feature-fusion MLPs use learning rate $z$6 with AdamW; NPO is trained for 10 epochs; experiments are repeated three times; and training is conducted on Nvidia H800 GPUs. For NavigScene generation, the reported hyperparameters are $z$7 sampled frames, $z$8 candidate descriptions, and weights $z$9, $N$0, and $N$1.

5. Reported performance and empirical interpretation

The central empirical claim is that navigation guidance improves not only QA outputs but also downstream driving behavior, including planning, safety, and cross-city generalization (Peng et al., 7 Jul 2025).

On DriveLM-nuScenes, all three VLMs improve after adding NavigScene. For Qwen2.5-7B, BLEU-4 rises from 51.65 to 55.13, METEOR from 34.12 to 38.20, CIDEr from 2.46 to 3.14, GPT score from 71.29 to 74.87, and Completeness from 30.31 to 34.26. Comparable improvements are reported for Llama-Adapter and LLaVA-7B.

On NuInstruct, the gains cover perception, prediction, and planning. For Qwen2.5-7B, the planning score rises from 26.6 to 36.4; the “closest object” score improves from 26.5 to 33.7; “same-road detection” improves from 26.7 to 31.9; and motion prediction error drops from 7.8 to 3.8. The pattern is reported as consistent across all three backbones.

On open-loop and closed-loop driving, the improvements are more directly operational. For SparseDrive + Qwen2.5-7B, adding NavigScene reduces open-loop average L2 from 1.01 to 0.76 and open-loop average collision rate from 32.48 to 20.71; in closed loop, PDMS rises from 82.4 to 86.5, DAC from 91.7 to 96.0, and EP from 77.9 to 81.7. For VAD + Qwen2.5-7B, open-loop average L2 decreases from 1.22 to 0.99, collision rate from 43.30 to 32.41, and closed-loop PDMS from 80.6 to 84.0.

The paper also reports gains in perception and forecasting. With SparseDrive + Qwen2.5-7B, Detection mAP improves from 0.42 to 0.46, Tracking AMOTA from 0.39 to 0.45, Mapping mAP from 55.1 to 57.6, motion forecasting mADE from 0.62 to 0.58, and motion forecasting EPA from 0.482 to 0.498.

Ablations separate the contributions of NSFT and NPO. On DriveLM with Qwen2.5-7B, CIDEr progresses from 2.46 in the baseline to 2.77 with NSFT only, 2.82 with NPO only, and 3.14 with both. On NuInstruct planning, the sequence is 26.6, 29.9, 29.7, and 36.4. The end-to-end pattern is similar: for SparseDrive + Qwen2.5-7B, open-loop average L2 changes from 1.01 to 0.95 with NSFT only, 0.98 with NPO only, and 0.76 with both. This suggests that NSFT adapts the model to route-aware conditioning, while NPO further refines preference structure and generalization.

Cross-city transfer results are also reported. For VAD + Qwen2.5-7B, Boston→Singapore L2 drops from 0.97 to 0.70 and collision from 27.51 to 22.55; Singapore→Boston L2 drops from 0.81 to 0.61 and collision from 21.85 to 18.46. For SparseDrive + Qwen2.5-7B, Singapore→Boston collision decreases from 35.64 to 19.66. The paper interprets these as evidence that navigation-aware preference optimization improves generalization to unseen urban environments.

6. Relation to adjacent research and terminological issues

NavigScene belongs to a broader line of work that augments local perception with explicit global or contextual scene structure. In indoor vision-and-language navigation, SpatialNav introduces a Spatial Scene Graph (SSG) constructed after pre-exploration, with hierarchical floor, room, and object nodes and retrieval-based use of global layout and semantics during zero-shot navigation (Zhang et al., 11 Jan 2026). In text-goal instance navigation, Context-Nav treats the full caption as a global exploration prior via dense text-image alignment and then performs explicit viewpoint-aware 3D verification before accepting a candidate target (Jang et al., 10 Mar 2026). These systems address different tasks, but they share with NavigScene the general principle that local sensing alone is often insufficient for disambiguation and long-horizon decision making.

A common terminological confusion concerns the use of the string “NavigScene” in unrelated work. In “DivScene: Benchmarking LVLMs for Object Navigation with Diverse Scenes and Objects”, the paper explicitly states that “NavigScene” is not a named benchmark, dataset, or agent in that work. There, the closest referent is the task of navigating to diverse target objects in a large number of scene types, benchmarked through DivScene and DivTraj, not an official entity called NavigScene (Wang et al., 2024). In contrast, the 2025 autonomous-driving paper uses NavigScene as the formal name of an auxiliary navigation-guided dataset and a family of navigation-guided learning paradigms.

The stated limitations of the NavigScene framework are correspondingly narrow. The paper notes that future work should integrate dynamic navigation information and multi-modal navigation information. This suggests that the current formulation is route-centric and comparatively static: it effectively simulates map-app-style guidance, but does not yet represent the full richness of live traffic, rerouting, or broader multimodal map cues. A plausible implication is that its present contribution is best understood as a controlled first step toward route-aware, beyond-visual-range driving intelligence rather than a complete navigation stack.