TravelUAV Benchmark: UAV Navigation & Planning

• TravelUAV Benchmark is a comprehensive suite integrating datasets, simulators, models, and evaluation protocols for UAV-based perception, navigation, and task assignment.
• It features extensive vision-language navigation tasks with 12,149 flight episodes across diverse, photorealistic urban and semi-urban environments.
• The benchmark provides rigorous evaluation metrics and baseline algorithms for UAV navigation, object detection, tracking, multi-agent planning, and video coding tasks.

The TravelUAV Benchmark denotes a set of datasets, simulators, mathematical models, and evaluation protocols developed to advance algorithmic research for UAV-based perception, navigation, task assignment, and data analysis. It provides standardized platforms covering vision-language navigation, object detection and tracking, multi-agent planning, learned video compression, and embodied intelligence evaluation. TravelUAV enables reproducible comparison of baseline and state-of-the-art algorithms across navigation, vision, control, and assignment tasks under realistic aerial scenarios, diverse sensory modalities, and operational constraints.

1. Dataset Composition and Modalities

TravelUAV, as used in recent vision-language navigation research (Lin et al., 9 Nov 2025), is instantiated as a large-scale AirSim-based simulator corpus. It consists of 12,149 flight episodes across 20 photorealistic outdoor environments (urban and semi-urban), rendered with physics-based UAV dynamics and low-level continuous control. Each trajectory is annotated with a navigation goal: a target object chosen from 76 classes and a natural-language instruction (e.g., “Fly to the red water tower behind the tree”). Modalities include:

• RGB first-person camera images per decision step
• 3D agent state (position $\{x, y, z\}$ and orientation $\{\theta, \phi, \psi\}$)
• Dense ground-truth waypoints (3D coordinates + orientation)
• Language instructions

The benchmark is partitioned into canonical splits:

Split Name	Trajectories	Environments	Objects
Train	9,152	20 “seen” maps	76 classes
Test-Seen	1,410	20 (same as train)	76 classes
Test-Unseen-Map	958	2 new maps	76 classes
Test-Unseen-Object	629	20 maps	“unseen” objects

Episodes are classified “Easy” (<250 m) or “Hard” (≥250 m) by path length. To evaluate data efficiency, a constrained regime uses only 25% of trajectories for training (sampled per scene under fixed seed).

2. Benchmark Task Definitions

TravelUAV covers a range of task formalizations:

Vision-Language Navigation for UAVs

The main navigation objective is learning a policy $\pi_\theta(a_t | s_t, I)$ that maps:

• Current UAV state $$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$
• Natural-language instruction $I$ to a continuous action $a_t$ (translation/heading adjustment).

Episode termination occurs on reaching a spatial threshold (e.g., within 3 m of the goal) or the max horizon ($T=200$ steps).

Planning adopts a two-stage hierarchy:

1. Waypoint predictor/action decoder: proposes 3D local waypoints
2. Continuous-control engine (VLN-CE): interpolates low-level controls between waypoints

Hard episodes (≥250 m) specifically challenge long-horizon trajectory synthesis and temporal credit assignment.

Multi-UAV Task Assignment

TravelUAV also includes benchmarks for multi-UAV task allocation under the Extended Team Orienteering Problem (ETOP) (Xiao et al., 2020). Mathematically:

• Graph $G=(V,A)$, depot node $0$, fleet $K$, location nodes $\{\theta, \phi, \psi\}$0
• Distances $\{\theta, \phi, \psi\}$1, rewards $\{\theta, \phi, \psi\}$2, service times $\{\theta, \phi, \psi\}$3, UAV speeds $\{\theta, \phi, \psi\}$4, time budget $\{\theta, \phi, \psi\}$5
• Decision variables: $\{\theta, \phi, \psi\}$6 (UAV $\{\theta, \phi, \psi\}$7 visits node $\{\theta, \phi, \psi\}$8), $\{\theta, \phi, \psi\}$9 (UAV $\pi_\theta(a_t | s_t, I)$0 traverses $\pi_\theta(a_t | s_t, I)$1)

Objective:

$\pi_\theta(a_t | s_t, I)$2

Subject to node, route, and time constraints (flow, binary integrality, subtour elimination).

Scales: Small ($\pi_\theta(a_t | s_t, I)$3, $\pi_\theta(a_t | s_t, I)$4), Medium ($\pi_\theta(a_t | s_t, I)$5, $\pi_\theta(a_t | s_t, I)$6), Large ($\pi_\theta(a_t | s_t, I)$7, $\pi_\theta(a_t | s_t, I)$8).

3. Evaluation Protocols and Metrics

TravelUAV prescribes multiple rigorous metrics for fair comparison of algorithms:

Navigation Metrics

• Success Rate (SR):

$\pi_\theta(a_t | s_t, I)$9

• Oracle Success Rate (OSR):

$$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$0

• Success weighted by Path Length (SPL):

$$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$1

where $$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$2 indicates success, $$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$3 actual, $$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$4 shortest path.

• Normalized Error (NE):

$$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$5

Detection and Tracking Metrics (Du et al., 2018)

• Object Detection: Precision, Recall, Average Precision (AP @ IoU=0.7)
• Single-Object Tracking: Success (AUC), Precision (center-error ≤ 20 px)
• Multiple-Object Tracking (MOTA):

$$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$6

with MOTP, IDF1, MT/ML, FP/FN/IDSW/FM as secondary metrics.

Video Coding Metrics (Jia et al., 2023)

• PSNR/bpp curves
• BD-Rate ($$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$7BD-rate):

$$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$8

for rate-distortion comparison.

Task Assignment Metrics

• Total Reward Collected
• CPU Time (wall-clock)
• Variance, statistical significance over multiple runs

4. Baseline Algorithms and Experimental Configuration

Standardized baselines are provided for each task domain:

Vision-Language Navigation

• Random policy: Uniform control sampling
• Fixed policy: Hand-coded waypoint sequence
• Cross-Modal Attention (CMA): Bi-LSTM with attention over image and text
• TravelUAV (state-of-the-art UAV-VLN): Hierarchical planning, rule-based policy

Training uses 4×NVIDIA RTX-4090, Adam optimizer (lr=$$s_t \in \mathcal{S} = \{x, y, z, \theta, \phi, \psi\}$$9), batch size=1, RL horizon $I$0.

Three assistance levels:

• L1: Instructions + Helper waypoints
• L2: Helper waypoints only
• L3: No assistance (autonomous)

Multi-UAV Task Assignment (Xiao et al., 2020)

• Genetic Algorithm (GA): Permutation + split-vector encoding, fitness by collected reward, elitist survival
• Ant Colony Optimization (ACO): 2D pheromone table, heuristic $I$1, evaporation/update by reward
• Particle Swarm Optimization (PSO): Particle encodes route split, velocity updates, sorting-based decoding, local swarms

Reference configurations and open-source implementations are available for reproducibility.

Object Detection and Tracking (Du et al., 2018)

• Detectors: Faster R-CNN, R-FCN, SSD, RON
• Trackers: MDNet, ECO, GOTURN, SORT, DSORT, MDP

Learned Video Coding (Jia et al., 2023)

• HEVC-SCC (anchor)
• OpenDVC
• MPAI EEV (enhanced OpenDVC): Two-stage residual modeling, in-loop restoration

5. Reward Shaping, Ablations, and Analysis

TravelUAV incorporates advanced reward shaping for RL-based navigation:

• Value-model reward:

$I$2

encourages monotonic state value progression

• Cosine-similarity reward:

$I$3

where $I$4, $I$5 are multimodal embeddings

Reward caps ($I$6) govern gradient stability. Ablation studies indicate $I$7 achieves optimal gradient informativeness.

Key quantitative results under the 25% data regime:

• OpenVLN vs. TravelUAV baseline: NE drops 132.59→125.97; SR +2.80pp (11.59%→14.39%); OSR +3.53pp (24.50%→28.03%); SPL +2.49pp (10.45%→12.94%)
• Hardest case (Hard/Test-Seen/L3): SR=0.00% (TravelUAV) vs. 0.47% (OpenVLN)

6. Difficulty Factors, Limitations, and Extensions

The TravelUAV Benchmark reveals unique challenges for UAV-centric vision and planning:

• High object density: Mean ≈10.5 objects/frame; endemic ID switches and FP in MOT
• Small/tiny object scale: Medium/high altitude imagery severely degrades detection/tracking
• Rapid camera motion/viewpoint change: Requires robust feature updating and domain-agnostic MC
• Adverse conditions: Diverse weather, occlusion, out-of-view rates stress model generalization
• Long-horizon navigation: Credit assignment over 250–400 m, sparse reward landscape

Video coding benchmarks indicate learned codecs excel in outdoor, high-motion regimes versus block-based HEVC, but lag on indoor/fisheye scenes without UAV-specific domain adaptation.

Suggested research directions include domain adaptation for video codecs, multi-scale feature fusion for tiny-object detection, real-time model pruning/quantization, semantic-aware compression, and robust physical simulation (wind, sensor perturbations).

The multi-UAV assignment benchmark provides a complete recipe for problem instance generation, permutation/split encoding, and comparative evaluation—including downloadable open-source code.

7. Significance and Research Utility

TravelUAV establishes a unified, extensible suite of protocols for evaluating algorithms within the UAV domain. Integration with platforms such as AirSim, Unreal, and open-source solvers, and prescriptive evaluation metrics ensure comparability and reproducibility. Its heterogeneous task set—vision-language navigation, detection/tracking, multi-agent planning, data-efficient RL—offers rich ground for algorithmic development and benchmarking under aerial robotics constraints.

OpenVLN’s demonstrated improvements (up to 4.34% SR, 6.19% OSR, 4.07% SPL over baseline methods (Lin et al., 9 Nov 2025)) exemplify the benchmark’s capacity to empirically validate advances in data-efficient, language-guided UAV navigation. The TravelUAV corpus continues to motivate research in long-horizon planning, robustness to adversarial conditions, domain adaptation, compute-energy-accuracy trade-offs, and the systematic evaluation of embodied UAV agents.