UAV-ON: Integrated UAV Benchmark & Middleware

• UAV-ON is a convergent UAV research domain that integrates vision-based benchmarks, real-time operating systems, and communication frameworks for intelligent aerial operations.
• The platform employs event-based vision, distributed AI middleware, and rigorous performance metrics to evaluate object-goal navigation and onboard target detection in 3D environments.
• Advanced optimization frameworks in UAV-ON drive energy-efficient cooperative mobility and robust ISAC, ensuring real-time performance and scalable multi-drone coordination.

Unmanned Aerial Vehicle Open Network (UAV-ON) refers to a convergent domain in UAV research that encompasses benchmarking, operating systems, communications, autonomy, and cooperative mobility for intelligent aerial vehicles. In current literature, "UAV-ON" denotes three primary lines: (i) experimentally grounded datasets and evaluation frameworks for perception and navigation, (ii) operating system and distributed AI middleware for real-time multi-drone operations, and (iii) optimization and network-theoretic policies for energy-efficient, large-scale UAV mobility and service orchestration. Recent works draw on event-based vision, open-world goal navigation, cooperative ISAC (Integrated Sensing and Communication), and rigorous platform-level performance.

1. Benchmarks for Perception and Semantic Navigation

The UAV-ON designation is used for aerial agent benchmarks supporting object-goal navigation (ObjectNav) in open-world environments and for robust onboard event-based tiny target detection.

ObjectNav Benchmark:

The UAV-ON Object Goal Navigation benchmark (Xiao et al., 1 Aug 2025) formalizes the object goal navigation task for UAVs in large, unstructured 3D environments. The agent operates within a 6-DoF state space $$s_t = (x_t, y_t, z_t, \phi_t, \psi_t, \theta_t) \in \mathbb{R}^3 \times SO(3)$$—typically reduced to position and yaw under flight constraints. Actions are continuous-magnitude movement primitives over ascending, descending, translation, and rotation. Observation input is four-view RGB-D imagery ($$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$) with no GPS or map.

Each navigation episode is specified by an instance-level prompt $c = (\text{name}, \text{size}, \text{description})$, combining category, 2D footprint, and detailed visual/semantic descriptors. Success is realized if the agent issues STOP within 20-simulation-unit Euclidean distance to the ground-truth object. Success Rate (SR), Oracle Success Rate (OSR), Distance to Success (DTS), and Success-weighted Path Length (SPL) are the core evaluation metrics.

Empirical results indicate that state-of-the-art baselines (CLIP-based heuristics and modular LLM-driven Aerial ObjectNav Agents) achieve low SR (4.2–7.3%) and very modest SPL, with all methods suffering from high collision rates (>30%), demonstrating the compounded challenge of semantic grounding and large-scale, obstacle-laden exploration in 3D aerial domains. Open problems include robust prompt-conditioned control, safety-aware RL, and sim-to-real transfer (Xiao et al., 1 Aug 2025).

Motion-on-Motion Event-Based Detection:

The M$^2$E-UAV ("UAV-ON") benchmark (Yan et al., 11 May 2026) defines tiny UAV detection from an onboard event camera in a regime where both observer and target are moving, fundamentally breaking the clean-background assumption of static or ground-observer setups. The dataset contains 87,223 training and 21,395 validation event-IMU packet pairs across diverse illumination and terrain scenarios, each event packet as $E = \{e_i\}_{i=1}^n$ with $e_i = (x_i, y_i, t_i, p_i)$.

Benchmarked methods span event-frame CNNs (YOLOv10), voxel-grid 3D vision transformers (RVT, SAST), and point-set networks (EV-UAV, KPConv, RandLA-Net, COSeg). RVT achieves $F_1$ score 0.8643 and $mAP_{50:95}$ of 0.3815—the highest, yet indicative of significant bounding box jitter and failure to precisely localize tiny targets under dense ego-motion-induced clutter. All architectures degrade in low-contrast, high-background-texture scenes (e.g., "sunset farm-village") (Yan et al., 11 May 2026).

2. Operating Systems and AI Middleware

UAV-ON also refers to a multi-layered, AI-accelerated operating system architecture (Tan et al., 2024), built on NVIDIA Orin hardware and a PREEMPT_RT–enhanced UNIX kernel supporting strict real-time constraints (⩽100 μs scheduling latency). The system encompasses:

• Distributed Dataflow: Pipe-and-filter, DDS-style publish/subscribe IPC spanning vision, perception, navigation, coordination, and ground station modules. State sharing and consensus for fleets is implemented via distributed averaging.
• Resource Management: Dynamic CPU/GPU task prioritization governed by a utility function $$U_i(t) = \alpha_i p_i/(D_i - t) + \beta_i \Delta \text{QoS}_i/\Delta \text{Load}$$, with enforcement via cgroups and GPU QoS zones.
• Interrupt Latency: Two-level interrupt stack achieving worst-case $\leq 78\,\mu$s under full load, with hardware ISR (⩽5 μs) and real-time scheduled bottom halves.
• Security/Fault Tolerance: Layered encryption (TLS 1.3, AES-256-GCM), RBAC/SELinux, triple modular redundancy for perception, watchdog coverage, overall $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$0 h (formally measured).
• Modularity/Extension: UA DevKit low-code framework enables algorithm drag-and-drop, simulation-in-the-loop via Gazebo/SITL, and hot deployment to flying hardware (Tan et al., 2024).

Benchmarks on Orin-class hardware confirm end-to-end closed-loop latency ≈30 ms and $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$199.7% mission success across 1,000 Monte Carlo trials in realistic urban scenarios.

3. Communication, Sensing, and ISAC in UAV-ON

UAV-ON approaches jointly consider communications, ISAC (Integrated Sensing And Communication), and trajectory optimization, providing on-demand detection and localization while maintaining user throughput (Yuan et al., 19 Dec 2025).

• System Model: A rotary-wing UAV flies a periodical fixed-altitude trajectory, discretized into $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$2 slots. Communication to $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$3 users follows the capacity: $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$4, with channel models validated by 3D aerial-to-ground measurements (Fuschini et al., 2021).
• On-demand Sensing: Detection SNR and localization CRB constraints impose regions ($$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$5) and path requirements. Formally, the radar-echo SNR at position $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$6 must exceed $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$7, and for localization, Fisher information matrices yield per-slot CRB bounds $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$8 on set $$o_t = \langle v_{t,\text{front}}^R, \ldots, v_{t,\text{down}}^D \rangle$$9 of sampled points.
• Optimization: The system maximizes minimum user throughput under communication, detection, localization, and kinematic constraints. A sequence of convex subproblems (via Successive Convex Programming) tightly majorize the nonconvex objective and constraints, with convergence to at least KKT points and demonstrated strict superiority over Taylor-descend and fixed-step baselines (Yuan et al., 19 Dec 2025).
• Deployment Guidelines: On-demand service is enabled by path restriction within convex regions ($c = (\text{name}, \text{size}, \text{description})$0) and adaptive reference points $c = (\text{name}, \text{size}, \text{description})$1 for localization constraints. Real deployments can tune the CRB threshold $c = (\text{name}, \text{size}, \text{description})$2 for fidelity/goodput tradeoff.

Experimental characterization (Fuschini et al., 2021) of millimeter-wave and UWB channels in urban canyons underpins link-budget planning. For 27 GHz, path-loss exponent $c = (\text{name}, \text{size}, \text{description})$3 (LOS, hover) to $c = (\text{name}, \text{size}, \text{description})$4 (NLOS, in-street), azimuth spread $c = (\text{name}, \text{size}, \text{description})$5; accurate main-lobe alignment, cm-level positioning, and adaptive beam steering are needed for robust aerial ISAC.

4. Cooperative Mobility and Resource-Oriented UAV-ON

The UAV-ON paradigm also covers energy-efficient mobility through cooperative "hitching" of UAVs on ground vehicles, optimizing for both range extension and battery/mission constraints (Ruan et al., 2021).

• System Model: $c = (\text{name}, \text{size}, \text{description})$6 UAVs each with mission distance $c = (\text{name}, \text{size}, \text{description})$7 and deadline $c = (\text{name}, \text{size}, \text{description})$8 may hitch on $c = (\text{name}, \text{size}, \text{description})$9 ground vehicles (with velocity $^2$0, angle $^2$1, and optional charge rate $^2$2). The optimal fraction of the trip ($^2$3) to hitch is analytically determined.
• Optimization: For single UAVs, closed-form rules decide vehicle eligibility and $^2$4 balancing energy and time via $^2$5. For fleets, global matching via the Max-Saving Algorithm (MSA) solves a capacity-constrained bipartite assignment problem, ensuring overall savings optimality.
• Key Results: Hitching plus charging vehicles allow UAVs to detour from direct routes when $^2$6 is high; slower vehicles can be preferred for more battery recharge duration. Fleet-wide, MSA improves total energy savings up to 20% over greedy policies, converging efficiently (Ruan et al., 2021).
• Deployment: Broadcast of mission and battery state, vehicle parameters, and utilization of local auctions/matching enable scalable, infrastructure-free UAV-ON mobility.

5. Real-Time Onboard Tracking and Visual Perception

Complementary to deep-learning-based benchmarks, classical real-time tracking systems have been demonstrated on embedded platforms (Qadir et al., 2012). The SUNDOG payload implements zero-mean normalized cross-correlation (ZMNCC) for object detection, with rotation invariance enabled by a bank of 36 pre-warped templates. Search-window restriction using EKF minimizes compute while guaranteeing rapid re-detection when targets re-enter the field of view, achieving $^2$71 pixel centroid error and $^2$825 fps on PC/104+ single-board computers—demonstrating a plausible pathway for low-latency, robust onboard perception in UAV-ON settings.

6. Open Challenges and Future Research Directions

Across its various instantiations, UAV-ON research identifies several persistent challenges:

• Disentangling target-movement cues from ego-motion-induced clutter in event-based vision (Yan et al., 11 May 2026).
• Training prompt-conditioned autonomous aerial agents for robust goal-directed navigation with semantic ambiguity (Xiao et al., 1 Aug 2025).
• Jointly optimizing trajectories to satisfy ISAC multi-QoS (detection, localization, communication), coping with the complexity of convex region constraints and high-dimensional reference points (Yuan et al., 19 Dec 2025).
• Maintaining strict real-time guarantees, high mission reliability, and extensibility in AI OS for coordinated drone fleets (Tan et al., 2024).
• Energy/range maximization via cooperative, opportunistic multi-vehicle routing under deadline, path, and charging constraints (Ruan et al., 2021).

Research is converging toward enhanced motion compensation via event-IMU integration, learning-based multimodal fusion, spatio-temporal grouping, contrastive/self-supervised pretraining, and scalable, auction-based coordination frameworks. Real-world deployments increasingly integrate RTK-GPS, multiband radios, advanced beamforming, and automated mission programming to close the gap from simulation and small-scale trials to robust, networked UAV-ON operations.

---

References:

• (Yan et al., 11 May 2026): "M$^2$9E-UAV: A Benchmark and Analysis for Onboard Motion-on-Motion Event-Based Tiny UAV Detection"
• (Xiao et al., 1 Aug 2025): "UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial Agents"
• (Tan et al., 2024): "An Integrated Artificial Intelligence Operating System for Advanced Low-Altitude Aviation Applications"
• (Ruan et al., 2021): "Optimal UAV Hitching on Ground Vehicles"
• (Yuan et al., 19 Dec 2025): "UAV-Enabled ISAC: Towards On-Demand Sensing Services and Enhanced Communication"
• (Fuschini et al., 2021): "An UAV-based Experimental Setup for Propagation Characterization in Urban Environment"
• (Qadir et al., 2012): "Implementation of an Onboard Visual Tracking System with Small Unmanned Aerial Vehicle (UAV)"