3D-LENS: A 3D Lifting-based Elevated Novel-view Synthesis method for Single-View Aerial-Ground Re-Identification

Abstract: Aerial-Ground Re-Identification (AG-ReID) is constrained by the viewpoint-domain gap, as drastic viewpoint disparities occlude or distort discriminative features, making cross-viewpoint image retrieval challenging. While existing methods rely on paired cross-view annotations, real-world deployments, such as wilderness search-and-rescue (SAR), often lack target-domain data, requiring retrieval from ground-level references alone. To our knowledge, we are the first to address this challenge by formalizing the Single-View AG-ReID (SV AG-ReID) setting, where models trained on a single real viewpoint must generalize to an unseen viewpoint. We propose 3D Lifting-based Elevated Novel-view Synthesis (3D-LENS), a unified framework combining geometrically-consistent novel view synthesis that leverages large-scale 3D mesh reconstruction, with a robust representation learning scheme to mitigate synthetic-to-real bias. Unlike 2D generative baselines that suffer from geometric inconsistencies or prior 3D methods that are restricted to class-specific templates, our approach ensures view-consistent synthesis across diverse categories without predefined templates that fail to capture fine-grained details, such as carried objects. Extensive experiments demonstrate that our method achieves state-of-the-art performance on SV AG-ReID scenarios. Code and data will be released at https://github.com/TurtleSmoke/3D-LENS.

• The paper introduces a method that reconstructs textured 3D meshes from single images to generate consistent synthetic views for aerial-ground re-identification without paired data.
• Employing calibrated rendering and a curriculum scheduler, the approach bridges the synthetic-real domain gap and improves mAP by up to 21.2 percentage points on benchmark datasets.
• Its class-agnostic design eliminates the need for template models, demonstrating scalability across diverse categories including humans, vehicles, and animals.

3D-LENS: Lifting-Based Synthesized Novel Views for Single-View Aerial-Ground Re-Identification

Overview and Problem Formulation

The paper "3D-LENS: A 3D Lifting-based Elevated Novel-view Synthesis method for Single-View Aerial-Ground Re-Identification" (2604.26520) introduces a formalization and solution for Single-View Aerial-Ground Re-Identification (SV AG-ReID), where the goal is cross-view re-identification without paired aerial and ground data in training. This scenario is relevant to practical deployments such as search-and-rescue or wildlife monitoring, where reference images exist only from a single viewpoint (e.g., ground camera), and models must generalize to unseen viewpoints (e.g., aerial UAV).

The core challenge is the extreme viewpoint-domain gap: discriminative features visible in ground-level images are occluded, distorted, or absent in aerial perspectives, and vice versa. Generative, domain adaptation, and template-based prior methods fail either due to geometric inconsistencies, class-specific limitations, or insufficiently structured synthesis, motivating the need for a class-agnostic, geometry-aware solution.

3D-LENS achieves this via large-scale 3D mesh reconstruction from single images—these meshes are class-agnostic and avoid manual attribute annotation or category-specific templates. By leveraging advances in 3D asset generation (e.g., Hunyuan3D), the method synthesizes geometrically-consistent novel views to span unseen camera perspectives, supplementing them with robust representation learning to bridge synthetic-real domain biases.

Related Work Context

Extant AG-ReID methods rely on paired cross-view supervision, soft-attribute annotation, view-invariant feature disentanglement, or 2D augmentations for simulating viewpoint variation. Typical generative methods, including diffusion and conditional GANs, fail to guarantee multi-view geometric consistency and often hallucinate deformities across synthesized views. Prior 3D re-identification work depends on parametric models (e.g., SMPL for humans, CAD models for vehicles), point clouds, or UV mappings, but suffer from category-bound generalization and breakdown in the presence of accessories (e.g., backpacks).

3D-LENS subsumes the strengths of geometric priors but transcends template restrictions to handle arbitrary object classes via learned 3D priors, synthesizing consistent views irrespective of category, occlusion, or carried objects.

Methodology

Geometrically-Consistent Novel View Synthesis

The pipeline proceeds in three stages:

1. 3D Mesh Reconstruction and Pose Calibration: Source images undergo foreground segmentation and are lifted to textured 3D meshes using a single-view 3D estimator. Camera pose optimization aligns the virtual and real perspectives via maximization of IoU between observed mask and rendered silhouette. Meshes below a quality threshold are discarded.
2. Novel View Rendering: Calibrated meshes enable rendering of images under controlled perturbations of azimuth and elevation. For ground-to-aerial synthesis, elevation is restricted to maintain texture fidelity (e.g., up to $30^\circ$). Mesh-based rendering ensures cross-view synthesis is structurally and visually coherent, in contrast to stochastic pixel generation.
3. Synthetic-to-Real Domain Alignment: Rendered foregrounds are composited onto real, inpainted backgrounds (using models like LaMa) and further processed via style transfer modules (e.g., StyleID) to harmonize illumination, boundary, and scene context, mitigating overfitting to synthetic artifacts.

Robust Representation Learning

A twofold strategy bridges the domain gap:

• Curriculum Scheduler: Difficulty is automatically scaled with elevation perturbation magnitude, modulating sample acceptance probability over training epochs. Near-source synthesized views are prioritized early, reducing the risk of destabilizing optimization.
• Balanced Sampling: Synthetic and real samples are mixed in mini-batches at fixed ratios, preventing saturation by synthesized views. Epochs are defined over the real dataset, ensuring all real images are seen, while synthetic views fill prescribed slots.
• Loss Functions: The backbone is a ViT with an appended domain token. The total loss is a combination of cross-entropy, triplet, and domain classification terms. The domain token is supervised to decouple domain cues from identity features, further reducing synthetic bias.

Experimental Results

Comprehensive evaluation spans AG-ReID [nguyen2023aerial], AG-ReID.v2 [nguyen2024ag], and the MOO cattle dataset [grolleau2026moomultivieworientedobservations], covering person and animal re-identification across aerial, ground, wearable, and synthetic views.

• On AG-ReID, 3D-LENS achieves state-of-the-art mAP improvements, e.g., +14.3 percentage points over the strongest baseline (DCAC) in ground-to-aerial evaluation and +21.2 pp on AG-ReID.v2 for CCTV-to-aerial. Rank-1 improvements are similarly pronounced.
• On MOO, the method outperforms the best template-based approaches (e.g., RotTrans) by 10 mAP points, demonstrating generalization beyond humans.

Ablation studies isolate the contributions of each module. Raw novel view synthesis impairs performance; background compositing, style transfer, balanced sampling, and curriculum scheduling each incrementally recover and enhance accuracy. The full framework yields up to +6.4 mAP on $A \rightarrow G$ and +5.3 mAP on $G \rightarrow A$ relative to strong baselines.

Practical and Theoretical Implications

3D-LENS's class-agnostic geometric approach enables AG-ReID deployments absent paired data, alleviating the need for costly annotation or template engineering. The explicit 3D lifting and calibrated rendering guarantee view-consistent supervision, making synthetic augmentation more reliable. This positions large-scale 3D mesh generation as a scalable alternative for cross-view generalization problems in ReID, applicable to arbitrary categories (vehicles, animals, etc.).

However, synthetic views remain bounded by the source image quality (occlusion, resolution, ambiguity), and artifacts in mesh reconstruction can propagate through rendering. As such, synthetic data cannot be equated to real data for supervision—quality control and weighting mechanisms in training are requisite future directions.

Speculation and Future Directions

Open-world 3D reconstruction will likely underpin further advances in controllable, geometry-aware data generation for re-identification and related tasks (e.g., pose-invariant retrieval, cross-modal search). Automated mesh filtering and reliability metrics could become standard for synthetic augmentation. Cross-view representation learning may further benefit from explicit geometric constraints, adversarial domain adaptation, and multi-modal (e.g., depth, LiDAR) integration.

Advancements in mesh reconstruction fidelity and efficiency will be crucial: as single-view 3D synthesis matures, the trade-off between synthetic scalability and real-world generalization will tip further toward synthetic supervision.

Conclusion

3D-LENS formalizes and solves SV AG-ReID, demonstrating that class-agnostic 3D mesh generation and calibrated rendering can bridge extreme viewpoint gaps without paired supervision. The holistic integration of geometry-driven synthesis and robust representation learning achieves superior performance in person and animal re-identification tasks across real and synthetic datasets. The method sets a strong precedent for geometry-aware augmentation, but also highlights the ongoing need for quality-controlled synthetic data and reliable adaptation mechanisms. The approach, while not universally substituting real data, establishes 3D lifting and rendering as essential tools for future AI deployments constrained by domain data scarcity.