AeroEye-v1.0: Scene Graph Benchmark & UAV Workflows

• AeroEye-v1.0 is an aerial VidSGG dataset with 2,260 videos and 261,503 frames, annotated with 687 predicates across five interactivity types.
• The benchmark employs dense tracking and evaluation metrics (Recall, mR) to assess dynamic scene graph generation under varying aerial and oblique viewpoints.
• Beyond dataset curation, AeroEye-v1.0 also defines modular UAV workflows for tracking, scanning, and micro-inspection using low-cost, real-time platforms.

Searching arXiv for the cited AeroEye-v1.0-related papers and closely related UAV/aerial vision work. AeroEye-v1.0 denotes an aerial-vision construct that is most explicitly defined as an aerial video dataset for Video Scene Graph Generation (VidSGG), introduced alongside THYME to address drone and oblique viewpoints in which objects are small, scales vary widely, and interactions are brief yet consequential (Nguyen et al., 12 Jul 2025). In recent engineering descriptions, the same label is also used for a low-cost real-time UAV visual intelligence platform and for an aerial micro-inspection system, extending its meaning from benchmark curation toward deployable perception-and-control stacks (Ungureanu et al., 2 Jul 2026, Mirtajadini et al., 9 Jun 2026). This suggests a dual role for AeroEye-v1.0: a concrete benchmark for dynamic scene understanding and an umbrella designation for modular aerial visual intelligence workflows.

1. Nomenclature and research scope

Source	Referent	Key properties
(Nguyen et al., 12 Jul 2025)	AeroEye-v1.0 dataset	Aerial VidSGG benchmark; 2,260 videos; 261,503 frames; five interactivity types
(Ungureanu et al., 2 Jul 2026)	AeroEye-v1.0 platform mapping	DJI Tello; offboard CPU server; Tracking, Scanning+Recognition, Line Following
(Mirtajadini et al., 9 Jun 2026)	AeroEye-v1.0 micro-inspection mapping	PX4; dual-camera payload; gimbal inspection; partitioned surface scanning

The most direct and publication-level definition is the dataset introduced in “THYME: Temporal Hierarchical-Cyclic Interactivity Modeling for Video Scene Graphs in Aerial Footage,” where AeroEye-v1.0 extends the earlier AeroEye dataset by adding five interactivity types, broadening predicate vocabularies, and maintaining temporal tracking (Nguyen et al., 12 Jul 2025). The engineering usages are explicitly framed as mappings from separate UAV systems into an AeroEye-v1.0 specification space rather than as original titles, so the dataset usage remains the primary referent.

A plausible implication is that AeroEye-v1.0 occupies two adjacent layers of aerial AI research. At the semantic layer, it is a richly annotated benchmark for dynamic scene graphs. At the systems layer, it names modular UAV workflows for tracking, scanning, navigation, and micro-inspection. The shared emphasis across these layers is temporally coherent visual understanding under aerial constraints.

2. Dataset composition and intended problem setting

AeroEye-v1.0 was introduced to advance VidSGG under drone and oblique viewpoints where objects are small, scales vary widely, and interactions are brief yet consequential (Nguyen et al., 12 Jul 2025). It was created to overcome limitations in prior datasets that either lack aerial perspective or do not provide sufficiently rich, temporally-aware annotations of interactivity. Relative to the original AeroEye, it fills gaps in appearance and situation labels and extends limited predicate differentiation. Relative to ASPIRe, which formalized the same five interactivity types for ground-view videos, AeroEye-v1.0 brings aerial, oblique, and ground perspectives together with densely labeled aerial footage and a much larger predicate inventory.

The dataset comprises aerial videos captured by drones and includes oblique and occasional ground perspectives. Environmental diversity spans urban, suburban, and rural scenes, while situations encode weather such as sunny and rainy conditions, time of day such as day and night, and events such as traffic congestion, accidents, and floods. The reported scale is 2,260 videos spanning 261,503 frames, with over 2 million bounding boxes and associated tracking annotations. Table-level comparisons report 57 object classes, 687 predicate classes, and 29 scene categories; the dataset text reports 56 unique object categories, so the literature explicitly preserves a 56/57 discrepancy rather than resolving it (Nguyen et al., 12 Jul 2025).

The predicate inventory is partitioned into 157 appearance predicates, 128 situation predicates, 135 position predicates, 142 interaction predicates, and 125 relation predicates, totaling 687 predicates. For annotation density, the paper additionally reports approximately 752K position annotations, approximately 318K interaction annotations, and approximately 178K relation annotations. Viewpoint diversity is summarized as aerial, oblique, and ground.

Several dataset attributes remain unspecified. Precise aerial platforms, sensor models, altitude ranges, camera angles, motion profiles, geospatial metadata, video duration ranges, resolutions, frame rates, and file formats are not specified. Official train/validation/test splits and task subsets such as SGDet, SGCls, and PredCls for video are also not specified. This absence is materially relevant because AeroEye-v1.0 is intended as a benchmark, yet some deployment-critical metadata are unavailable in the published description.

3. Annotation schema and the five interactivity types

AeroEye-v1.0 annotates every frame. Each object is represented by a bounding box and assigned appearance and situation annotations. Pairwise spatial relations are annotated as position; dynamic actions between objects are annotated as interaction; and functional or role-oriented associations are annotated as relation. Object identities are temporally linked through tracking annotations so that object IDs remain consistent across frames (Nguyen et al., 12 Jul 2025).

Appearance describes visual characteristics of individual objects, including type, color, size, and shape. Examples include “white sedan” and “long cargo truck.” Situation describes environmental and contextual conditions, including setting, weather, time, and events; examples include “urban night, rainy” and “rural day, accident.” Position captures pairwise spatial relations such as “in front of,” “behind,” “beside,” “near,” and “inside.” Interaction captures dynamic actions such as “crashing,” “towing,” “overtaking,” “approaching,” and “following.” Relation captures functional or role-oriented associations such as “assisting,” “escorting,” “guiding,” and “blocking.”

Temporal behavior differs across these interactivity types. Appearance is mostly static but can vary, as in lights turning on or off. Situation can shift over time, including day-to-night transitions. Position often evolves rapidly, as in “behind” changing to “alongside” during a chase. Interaction is explicitly short-lived and transient. Relation can span longer segments and often co-occurs with changing position and interaction.

A common misconception would be to equate the dataset itself with THYME’s architectural ideas. The paper explicitly states that AeroEye-v1.0 does not annotate hierarchical or cyclic structures; those are introduced by the THYME model during learning rather than by the annotation schema. Temporal consistency in the dataset is supported by tracking-based identity maintenance, while specific label consistency enforcement procedures beyond tracking are not specified. The paper also does not publish an exact schema, though a representative snippet contains video_id, viewpoint, scene_category, per-frame objects with IDs, boxes, categories, appearance and situation fields, interactivity-specific relation blocks, and object tracks spanning frame indices.

4. Benchmark protocol, THYME, and empirical characteristics

The reported evaluation protocol on AeroEye-v1.0 uses Recall and mean Recall at top-20, top-50, and top-100 predictions across the five interactivity types (Nguyen et al., 12 Jul 2025). The standard definitions given are

$$R@K = \frac{\#\ \text{correct predictions within top-}K}{\#\ \text{ground-truth instances}}$$

and

$\mathrm{mR@K} = \frac{1}{C}\sum_{c=1}^{C} R@K_c,$

where $C$ is the number of predicate classes and $R@K_c$ is Recall@K for class $c$. The paper also lists standard forms for IoU and mAP,

$$\mathrm{IoU}(B_p,B_{gt})=\frac{|B_p\cap B_{gt}|}{|B_p\cup B_{gt}|}, \qquad \mathrm{mAP}=\frac{1}{C}\sum_{c=1}^{C} \mathrm{AP}_c,$$

but explicitly notes that IoU and mAP are not reported as official metrics for AeroEye-v1.0. No dedicated temporal consistency metric or cyclic consistency measure is specified.

On AeroEye-v1.0, THYME reports the following results. At R/mR@20, Appearance is 16.52/0.68, Situation is 5.53/0.61, Position is 15.52/1.05, Interaction is 13.07/0.16, and Relation is 16.03/0.95. At R/mR@50, Appearance is 19.53/0.72, Situation is 7.57/0.76, Position is 26.03/1.45, Interaction is 24.53/1.22, and Relation is 26.53/1.74. At R/mR@100, Appearance is 21.07/0.83, Situation is 11.51/0.85, Position is 42.03/2.15, Interaction is 41.53/2.26, and Relation is 48.03/2.38. Relative to strong recent baselines, THYME improves Recall by approximately 2–3% on double-actor attributes—Position, Interaction, and Relation—and yields consistent mR gains, indicating better performance on rare predicates in the long tail.

The ablation results are designed to isolate the roles of hierarchical aggregation depth, cyclic attention, and temporal window size. Increasing hierarchical depth improves R@20/mR@20 for Appearance from 14.12 to 16.52 and from 0.60 to 0.68; for Position from 12.32 to 15.52 and from 0.87 to 1.05; for Interaction from 10.87 to 13.07 and from 0.12 to 0.16; and for Relation from 13.03 to 16.03 and from 0.85 to 0.95. Cyclic attention outperforms standard attention; for Position at R/mR@20, the comparison is 15.52/1.05 versus 13.42/0.92. Enlarging temporal window size increases R/mR, with the best behavior reported at 3/4 to full windows, underscoring the importance of long-range temporal modeling in aerial footage where interactions can be brief and occlusions frequent.

The usage guidance emphasizes a difficult aerial operating regime. Small object sizes at altitude, high viewpoint variance from oblique angles, motion blur from drone motion, and transient occlusions are identified as central pitfalls. Suggested strategies include multi-scale feature extraction and hierarchical aggregation for stabilizing small-object relations, cyclic temporal attention for retaining long-range context across occlusions, and predicate debiasing or mean-Recall–oriented training to address long-tail predicates. Specific preprocessing protocols are not specified, but the paper describes uniform or motion-adaptive frame sampling, resizing to detector input scales used by DETR-like backbones, ImageNet-style normalization, and per-frame detection with box linking via tracking as an effective practice.

5. Modular low-cost UAV visual intelligence under the same designation

A distinct usage of AeroEye-v1.0 describes an accessible, modular UAV-based visual intelligence platform that runs on low-cost hardware and offloads inference to a CPU-bound server (Ungureanu et al., 2 Jul 2026). The baseline UAV is a DJI Tello, using a front-facing single RGB camera with 720p streaming and no native depth. Communication is by WiFi to an offboard Python server, command and telemetry are sent over UDP via DJITelloPy, and the reported compute substrate is an offboard CPU-only laptop or PC, with average CPU load below 65% on a “standard laptop (AMD Ryzen7)” in integrated mode.

The architecture is organized into three operational modes: Tracking, Scanning+Recognition, and Line Following. FollowMe performs person tracking from visual cues, using bounding-box centroid and area to regulate yaw, lateral motion, and forward/backward distance via PID control. SkyScan uses monocular depth estimation with DepthAnything V2, ViT-B, to plan yaw sweeps, perceive walls and obstacles up to approximately 20 m, and trigger recognition on detected faces. LineTracker follows floor lines using a “virtual sensor matrix” derived from vision; a 45° mirror redirects the front camera to the floor without hardware modification of the drone. The web-facing and backend interfaces include live monitoring, mode switching, manual override, video streaming, telemetry, and face enrollment to Firebase.

The vision stack is explicitly modular. YuNet is used for face detection, FaceNet provides 128-D embeddings for face recognition with Euclidean distance matching,

$d(x,y)=\|x-y\|_2,$

and DepthAnything V2 provides dense relative depth maps for indoor scanning. Control uses PID,

$$u(t)=K_p e(t)+K_i\int e(\tau)\,d\tau + K_d \frac{de}{dt},$$

with temporal validation across consecutive frames to suppress false positives and oscillatory behavior. In FollowMe, the centroid and area of the detected box define normalized positional and distance-proxy errors. In LineTracker, the forward speed is adapted to curvature as

$v_{\mathrm{fwd}}=v_{\max}(1-\alpha |\kappa|).$

Reported performance is specific. FollowMe achieves tracking accuracy above 92% across 50 trials, average correction time below 300 ms per positional change, and reacquisition after occlusion below 2 s in 89% of cases. SkyScan reports YuNet detection success above 95%, FaceNet recognition accuracy of 97.4%, and effective indoor scanning up to approximately 20 m. LineTracker completes 10/10 routes, including sharp turns, intersections, and partial occlusions, with average centerline deviation below 15 px. Reported end-to-end latencies are below 100 ms for FollowMe, below 150 ms for SkyScan, and below 120 ms for LineTracker. The system is explicitly positioned as CPU-friendly, low-cost, and open-source.

The platform description also states its constraints. There is no onboard GPU, no depth sensor, and limited onboard compute; all inference is therefore offloaded. Battery duration is not explicitly reported, though the text notes typical Tello-class operations of approximately 10–13 minutes as a planning reference rather than as a measured paper metric. Thresholds for FaceNet matching are not numerically specified, and quantization or pruning are suggested as extensions rather than reported components.

6. Extensions, adjacent aerial systems, and unresolved issues

A second engineering mapping uses AeroEye-v1.0 for aerial micro-inspection rather than general tracking or scene-graph understanding (Mirtajadini et al., 9 Jun 2026). This system is PX4-based, uses ROS 2 with a Micro XRCE-DDS bridge for telemetry and a separate MAVLink node for gimbal control, and deploys a dual-camera payload: a zoomed, gimbal-mounted inspection camera and a wide-field-of-view stereo navigation camera. On the real platform, the inspection stack uses a Gremsy Pixy LR gimbal with a Sony ILX-LR1 camera, a 55 mm lens, and 4× digital zoom for an effective horizontal field of view of approximately 9.2°, while navigation uses a StereoLabs ZED 2 with horizontal field of view of approximately 110°. Surface acquisition is performed with a YOLO-based segmentor, partitioning is scaled by camera intrinsics, feedback is provided by an OpenCV Median Flow tracker, and the gimbal visits partitions with a dwell time of approximately 2 s. In real-world experiments, the reported average pointing error is approximately 2.17° on a mockup tree and approximately 3.34° on greenhouse sticky traps.

The broader aerial-robotics context clarifies what AeroEye-v1.0 is not. Gesture-based monocular piloting on a DJI Matrice 100 uses DSST tracking, histogram-based skin detection, a 60-frame temporal buffer, and a 600 ms command gate to convert arm gestures into piloting commands; it is an economical natural user interface rather than a scene-graph or inspection framework (Sun et al., 2018). AU-AIR is a multi-modal low-altitude traffic-surveillance dataset with 8 video clips, over 2 hours of recording, 32,823 extracted and labeled frames, 132,034 annotated instances, and sensor metadata including time, GPS, altitude, IMU rotation, and velocity; its YOLOv3-Tiny baseline reaches mAP 30.22 at approximately 17.5 FPS on Jetson TX2 (Bozcan et al., 2020). A separate RGB-D landing-and-tracking system combines a LoG-based deep network with an iterative multi-model filter, reporting detection at up to 45 FPS and a reduction of maximum tracking error from 3.85 cm with an original Kalman filter to 1.57 cm with IMM (Liu, 2022). In yet another control lineage, image-based visual servoing for probe-and-drogue autonomous aerial refueling directly regulates 2D image errors and is reported to remain successful under a camera installation error of $\Delta p_c^r = [1, 0, -0.5]^T$, whereas PBVS fails in the reported simulation (Quan et al., 2023).

Across these uses, several unresolved issues recur. For the dataset, access URL, licensing terms, intended use policy, quality-control procedures, inter-annotator agreement, per-class histograms, long-tail analysis, average track lengths, occlusion metrics, and scale variability metrics are not specified (Nguyen et al., 12 Jul 2025). Ethical and privacy considerations are noted as important in aerial data but are not specified. For the low-cost UAV platform, onboard autonomy remains limited by Tello hardware, and WiFi/UDP link constraints require avoidance of command saturation (Ungureanu et al., 2 Jul 2026). For the micro-inspection system, stereo baseline and focal length for the real platform are not specified in the paper, and whitefly discrimination on sticky traps is explicitly identified as insufficient at the chosen zoom (Mirtajadini et al., 9 Jun 2026).

Taken together, AeroEye-v1.0 sits at a technically significant junction of aerial semantics and aerial autonomy. In its primary sense, it is a comprehensive aerial VidSGG dataset with five interactivity types, 687 predicate classes, viewpoint diversity, and dense temporal tracking. In its mapped engineering senses, it denotes modular UAV systems that operationalize tracking, scanning, navigation, and high-detail inspection. The common denominator is temporally structured visual intelligence under aerial sensing conditions where small-object perception, viewpoint shifts, occlusion, and control latency remain first-order design constraints.