AG-VPReID.VIR: Cross-Modality Video Re-ID Benchmark

• The paper introduces AG-VPReID.VIR as the first aerial–ground cross-modality video-based person re-ID benchmark combining RGB and infrared modalities.
• It leverages multi-platform data from UAVs, fixed CCTV, and wearable cameras to address viewpoint, modality, and temporal discrepancies in 24-hour surveillance.
• TCC-VPReID employs a three-stream architecture integrating style-robust feature learning, cross-view adaptation, and intermediary-guided temporal learning with advanced contrastive losses.

AG-VPReID.VIR is a video-based person re-identification benchmark and method paper that targets the joint problem of aerial-ground matching and visible-infrared matching in a single surveillance setting. It is defined as the first aerial-ground cross-modality video-based person Re-ID dataset, capturing 1,837 identities across 4,861 tracklets and 124,855 frames using UAV-mounted and fixed CCTV cameras in RGB and infrared modalities, and it is accompanied by TCC-VPReID, a three-stream architecture for cross-platform cross-modality video-based person Re-ID (Nguyen et al., 24 Jul 2025). The benchmark is explicitly framed around 24-hour surveillance, where UAVs mitigate occlusion and coverage limitations of fixed ground cameras, while infrared sensing supports night-time operation.

1. Historical placement and nomenclature

AG-VPReID.VIR emerged from two previously separate research lines: aerial-ground Re-ID and visible-infrared Re-ID. The earlier image-based aerial-ground benchmark AG-ReID established the cross-platform setting with 21,983 images of 388 identities, collected by a UAV flying at altitudes between 15 to 45 meters and a ground-based CCTV camera, and annotated with 15 soft attributes per identity (Nguyen et al., 2023). A later RGB-only video benchmark, AG-VPReID, expanded the scale to 6,632 subjects, 32,321 tracklets, and over 9.6 million frames captured by drones, CCTV, and wearable cameras, with UAV altitudes ranging from 15 to 120 meters (Nguyen et al., 11 Mar 2025).

Within the visible-infrared literature, widely used datasets such as SYSU-MM01, RegDB, HITSZ-VCM, and BUPTCampus are ground-based only, whereas aerial-ground datasets such as AG-ReID.v2 and G2A-VReID are RGB-only. AG-VPReID.VIR is introduced precisely to close that gap by combining UAV and ground viewpoints with RGB and infrared video in one benchmark (Nguyen et al., 24 Jul 2025).

The naming requires care. The 2025 aerial-ground challenge paper uses closely related terminology for a high-altitude subset of AG-VPReID and describes its core benchmark dataset as AG-VPReID.VIR or the high-altitude AG-VPReID subset, with 3,027 identities, 13,511 tracklets, and approximately 3.7 million frames captured from UAVs, CCTV, and wearable cameras at 80 m and 120 m (Nguyen et al., 28 Jun 2025). In the dedicated dataset paper, however, AG-VPReID.VIR denotes the visible-infrared aerial-ground cross-modality benchmark centered on RGB and IR video (Nguyen et al., 24 Jul 2025). This suggests that the literature contains a nontrivial naming overlap, and the term should be interpreted from the paper context rather than from the acronym alone.

2. Dataset composition and acquisition

AG-VPReID.VIR is organized around multiple camera platforms and sensing modalities rather than around a conventional fixed ground-camera network. The collection includes UAV-mounted cameras, fixed CCTV cameras, and wearable devices for additional ground-view RGB coverage. The modalities covered are RGB and infrared / NIR / thermal (Nguyen et al., 24 Jul 2025).

The paper reports the following camera setup.

Platform/modality	Device	Capture specification
CCTV RGB	Bosch	$704 \times 480$, 15 FPS, about 3 m
Wearable RGB	Vuzix M4000	4K, 30 FPS, about 1.5 m
UAV RGB	DJI XT2	$3840 \times 2160$, 30 FPS, altitude 15–45 m
UAV IR (NIR)	DJI XT2	$640 \times 512$, 30 FPS, altitude 15–45 m
CCTV IR (NIR)	Avigilon	$800 \times 600$, 30 FPS, about 4 m

Data collection spans 5 months and 10 sessions. UAVs flew at 15 m, 25 m, and 45 m, and each flight lasted 15 minutes. The annotation pipeline uses YOLOv10x for detection/tracking, followed by manual review and correction of tracklets, with unique identity labels consistent across all modalities and platforms (Nguyen et al., 24 Jul 2025).

The result is a cross-platform, cross-modality, sequence-level dataset rather than a frame-level image benchmark. That distinction is important technically: the retrieval unit is a tracklet, and the relevant nuisance factors include temporal instability and trajectory variation in addition to appearance variation.

3. Evaluation protocols and sources of difficulty

The benchmark is difficult because it composes three major discrepancies that are often studied separately: viewpoint discrepancy, modality discrepancy, and temporal discrepancy. The paper identifies four principal factors: cross-viewpoint variation, modality discrepancy, temporal dynamics, and scene complexity with occlusion (Nguyen et al., 24 Jul 2025).

Cross-viewpoint variation arises because aerial and ground cameras observe people from different geometry: top-down or oblique aerial views versus frontal or side ground views, with severe pose and perspective changes and different visible body parts. Modality discrepancy arises because RGB contains color and texture, whereas IR mainly captures thermal or emission patterns. Temporal dynamics matter because the dataset is video-based and includes motion changes over time, unstable trajectories, and temporal misalignment across camera views. Scene complexity adds partially occluded subjects, people walking in groups, cluttered campus scenes, and varying poses and distances.

The paper quantifies the scale problem directly. UAV-captured crops can range roughly from 31×59 to 371×678, while ground crops range from 22×23 to 172×413 (Nguyen et al., 24 Jul 2025). These dimensions indicate that the benchmark includes both very small aerial targets and substantially larger ground observations, which sharply increases the difficulty of learning stable identity cues.

The official split and protocol emphasize cross-platform and cross-modality retrieval. The training set contains 326 identities, 978 tracklets, and 24,793 frames. Testing includes four settings: Ground-to-Ground, Aerial-to-Aerial, Ground-to-Aerial, and Aerial-to-Ground. Each setting is evaluated in both directions: V2I, where the query is visible and the gallery is infrared, and I2V, where the query is infrared and the gallery is visible. For I2V, the gallery includes 1,184 extra distractor identities, making the task more realistic and harder (Nguyen et al., 24 Jul 2025).

A common misconception is to treat AG-VPReID.VIR as merely a visible-infrared extension of a ground-camera benchmark. The dataset is instead defined by the simultaneous presence of aerial-ground geometry, RGB-IR sensing, and sequence-level temporal structure. The paper explicitly argues that this combination changes the task definition relative to prior datasets (Nguyen et al., 24 Jul 2025).

4. TCC-VPReID architecture

The companion method, TCC-VPReID, is a three-stream architecture for Cross-platform Cross-modality Video-based Person Re-ID. Its three streams are Style-Robust Feature Learning, Memory-based Cross-View Adaptation, and Intermediary-Guided Temporal Learning. These streams are fused into a final representation for matching, and the full objective is

$$L_{total} = L_{id} + \lambda_1 L_{tri} + \lambda_2 L_{SA} + \lambda_3 L_{cr} + \lambda_4 L_{V2M},$$

with the best reported setting

$$\lambda_1 = 1.0,\quad \lambda_2 = 1.5,\quad \lambda_3 = 1.0,\quad \lambda_4 = 1.5$$

(Nguyen et al., 24 Jul 2025).

The first stream, Style-Robust Feature Learning, addresses appearance distortions, style shifts, intra-/inter-modal variation, and platform-specific visual changes. It uses channel-wise perturbation for RGB and IR:

$$I_t^{aug} = [\alpha I_t^R, \beta I_t^G, \gamma I_t^B], \qquad I_t^{ir,aug} = \delta I_t^{ir},$$

where $\alpha,\beta,\gamma,\delta \sim U(0.5, 1.5)$. It further applies an intra-modal style attack at the feature level,

$$\tilde{f}_{conv3}^{i,j} = \frac{\sigma(f_{conv3}^{k,l})}{\sigma(f_{conv3}^{i,j})} \cdot \left(f_{conv3}^{i,j} - \mu(f_{conv3}^{i,j})\right) + \mu(f_{conv3}^{k,l}),$$

and optimizes

$L_{SA} = L_{dis} + L_{con}.$

The second stream, Memory-based Cross-View Adaptation, is designed around the aerial-ground viewpoint gap. Instead of a single identity memory, it maintains $3840 \times 2160$0 and $3840 \times 2160$1, then refines them through a cross-platform prompt update,

$3840 \times 2160$2

where $3840 \times 2160$3. The principal supervision is a video-to-memory contrastive loss,

$3840 \times 2160$4

This stream is the explicit mechanism for platform-specific sequence memory and cross-view alignment.

The third stream, Intermediary-Guided Temporal Learning, targets the RGB-IR gap while preserving temporal structure. It constructs an anaglyph-based intermediate representation,

$3840 \times 2160$5

and narrows the modality gap with cross-reconstruction,

$3840 \times 2160$6

The implementation uses PyTorch, NVIDIA A100, Adam, a learning rate of $3840 \times 2160$7, cosine annealing, and 120 epochs, with a batch size of 8 identities × 4 sequences × 8 frames. The stream backbones are ResNet50 for the style-robust stream, CLIP ViT-B/16 with a 2-layer transformer decoder for the memory stream, and a dual-branch architecture with bi-LSTMs for the intermediary-guided stream (Nguyen et al., 24 Jul 2025).

5. Empirical behavior and ablation structure

On established visible-infrared video benchmarks, TCC-VPReID reports strong performance. On HITSZ-VCM it achieves 72.16% Rank-1 / 56.43% mAP for I2V and 75.92% Rank-1 / 57.84% mAP for V2I, exceeding SAADG by +2.94% Rank-1 / +2.66% mAP on I2V and +2.79% Rank-1 / +1.75% mAP on V2I. On BUPTCampus it achieves 69.73% Rank-1 / 67.25% mAP for I2V and 68.47% Rank-1 / 65.38% mAP for V2I, outperforming AuxNet by about 3.2%–3.3% in Rank-1 and mAP (Nguyen et al., 24 Jul 2025).

The AG-VPReID.VIR results are substantially lower, which the paper uses as evidence that the new benchmark is much harder. The reported cross-platform numbers are as follows.

Protocol	I2V Rank-1 / mAP	V2I Rank-1 / mAP
Aerial → Ground	19.83 / 22.61	46.54 / 59.69
Ground → Aerial	34.23 / 45.62	31.52 / 42.92
Overall best reported cross-platform numbers	36.18 / 41.56	46.33 / 59.23

The contrast with prior methods is stark. The paper notes that SAADG reaches around 69% Rank-1 on HITSZ-VCM but only 13.07% on AG-VPReID.VIR I2V, while MITML reaches about 63.74% on HITSZ-VCM but only 12.16% on AG-VPReID.VIR. This is presented as direct evidence that aerial-ground mismatch compounds the already difficult visible-infrared problem (Nguyen et al., 24 Jul 2025).

The ablation study attributes the strongest single-stream performance to St2, the memory-based cross-view adaptation stream; St1 follows; St3 is weakest alone. Combined systems improve further, and St123 yields the best overall results. The paper interprets this as evidence that cross-view identity consistency is especially important in aerial-ground Re-ID. It also reports an asymmetry: I2V is usually harder than V2I, because infrared queries lose more identity detail. Overly large loss weights hurt performance, so balanced multi-loss optimization is described as essential (Nguyen et al., 24 Jul 2025).

6. Research significance, adjacent directions, and open issues

AG-VPReID.VIR is significant because it is the first benchmark to combine aerial-ground viewpoints, RGB-IR modalities, and video-based person Re-ID in a single dataset. The paper further emphasizes that it introduces aerial infrared and presents a more realistic benchmark for 24-hour surveillance than either ground-only visible-infrared datasets or RGB-only aerial-ground datasets (Nguyen et al., 24 Jul 2025).

Its role becomes clearer when placed alongside adjacent benchmarks and methods. AG-ReID defined the elevated-viewpoint aerial-ground image problem (Nguyen et al., 2023). AG-VPReID generalized aerial-ground ReID to a large-scale RGB video setting with drones, CCTV, and wearable cameras (Nguyen et al., 11 Mar 2025). The AG-VPReID 2025 Challenge then pushed the RGB video regime to high-altitude aerial-ground matching at 80–120 m, where leading methods such as X-TFCLIP and TFCLIP-AG-VPReID used attention pooling, stronger CLIP backbones, memory mechanisms, and view-aware temporal reasoning (Nguyen et al., 28 Jun 2025). In a separate RGB-only aerial-ground line, SD-ReID argued that view-specific features are complementary rather than purely nuisance, and used Stable Diffusion to generate view-specific identity representations on AG-ReID benchmarks (Hu et al., 13 Apr 2025).

Against that background, AG-VPReID.VIR marks a stricter task composition: platform shift, modality shift, and temporal reasoning must be handled simultaneously. A plausible implication is that future progress will require models that combine cross-view adaptation, modality bridging, and sequence-level reasoning rather than treating any one of these as an auxiliary issue. The current empirical results support that reading: even a purpose-built three-stream architecture remains far below the performance typically reported on easier ground-only visible-infrared benchmarks, and the hardest asymmetries occur precisely where platform and modality shifts are compounded (Nguyen et al., 24 Jul 2025).