Cross-Platform 3D Perception

• Cross-Platform 3D Perception is a unified approach that integrates multi-modal sensor data from diverse platforms like vehicles, drones, and robots for comprehensive scene understanding.
• It leverages techniques such as platform-aware normalization and domain-adaptive fusion to mitigate sensor differences and environmental variations across agents.
• Advanced methods including collaborative depth optimization and self-training enhance detection, tracking, and semantic labeling for scalable deployment in automation and surveillance.

Cross-platform 3D perception refers to unified, generalizable approaches for 3D scene understanding across heterogenous platforms such as ground vehicles, UAVs, quadrupeds, humanoid robots, and distributed sensor networks. This paradigm addresses the fundamental challenge of integrating multi-modal sensory data (e.g., LiDAR, RGB, depth) from agents with distinct perspectives, sensor configurations, and operating constraints. The goal is to achieve robust detection, tracking, semantic labeling, and grounding that generalize across differing platforms and environments. Key drivers include collaborative autonomy, embodied intelligence, and scalable real-world deployment in domains ranging from automated driving and aerial surveillance to service robotics.

1. Platform and Sensor Diversity in 3D Perception

Cross-platform 3D perception brings together data from vehicles (street-level LiDAR/RGB), drones (aerial LiDAR/RGB), quadrupeds (low-mounted sensors with dynamic pitch/roll), humanoids (body-occluded, panoramic vision, LiDAR), and infrastructure nodes (roadside or static sensors). Datasets explicitly supporting this diversity include Pi3DET (vehicle, drone, quadruped, 563M points, (Liang et al., 23 Jul 2025)) and 3EED (vehicle, drone, quadruped, with 128k 3D objects and 22k referring expressions, (Li et al., 3 Nov 2025)).

Canonical challenges emerge from sensor placement (elevation shift, roll/pitch jitter), range-dependent sparsity (drones have 102 points/object; vehicles, 463), and morphology-induced distortions (e.g., panoramic self-occlusion in humanoids). Synchronized multi-modal data generation (RGB, depth, segmentation) at matched timestamps is handled in simulation environments such as TranSimHub (Wang et al., 17 Oct 2025), which supports causal scenario editing and top-down synchronization across air–ground agents.

2. Unified Representation and Domain-Adaptive Fusion

Cross-platform fusion requires normalizing sensor coordinates and representations to mitigate platform-induced domain gaps. Common strategies include:

• Platform-Aware Normalization (CPA): Rotation (R) to align gravity, altitude offset ($h_\beta$) for shared spatial reference, transforming each point as $\tilde p_i = R \cdot (p_i - h_\beta \cdot e_z)$ (Li et al., 3 Nov 2025).
• Random Platform Jitter (RPJ) and Cross-Jitter Augmentation (CJA): Simulate roll/pitch variations during source-domain training ($\Delta\phi, \Delta\theta \sim \mathcal{U}([-a, a])$) to enhance invariance (Liang et al., 23 Jul 2025, Feng et al., 13 Jan 2026).
• Virtual Platform Pose (VPP): Homogenize target scans by mapping actual ego-pose to canonical vehicle pose, with rotational compensation for box headings (Liang et al., 23 Jul 2025).

Feature fusion architectures include pillar-based encodings (VINet, (Bai et al., 2022); PillarGrid, (Bai et al., 2022)), BEV-based multi-modal fusion (UVCPNet, (Wang et al., 2024)), deformable transformers for local cross-modal sampling (CMDT, (He et al., 18 Apr 2025)), and spherical transformer networks for panoramic data (HumanoidPano, (Zhang et al., 12 Mar 2025)).

3. Cross-Domain Adaptation and Semi-/Unsupervised Learning

Robust cross-platform generalization relies on adaptation mechanisms:

• Self-training with pseudo-labeling (ST3D): Iteratively refine the detector using confident pseudo-labels from the unlabeled target domain. This demonstrably boosts Car [email protected] from 29.44% to 58.79% (quadruped) and to 62.67% (drone) without any target annotations (Feng et al., 13 Jan 2026).
• Adversarial scene- and instance-level feature alignment: Enforce indistinguishability of domain features via discriminators at both global and node (instance) levels (Zhang et al., 2023).
• KL Probabilistic Feature Alignment (PFA): Align latent distributions over region-of-interest features using explicit $D_{\text{KL}}$ regularization (Liang et al., 23 Jul 2025).
• Weak-label fine-tuning: Utilize category priors from heuristic pre-segmentation to guide unsupervised training when full target side annotation is unavailable (Zhang et al., 2023).

4. Multi-Modal Data Fusion, Depth Optimization, and BEV Projection

Accurate per-agent perception and inter-agent fusion are achieved through:

• Collaborative Depth Optimization (CDO): Refine monocular depth via a CRF leveraging semantic consistency across ground–aerial domains, minimizing the energy $E(d|S)$ for optimized BEV projection, directly boosting detection mAP by +3.1% (Wang et al., 2024).
• Cross-Domain Cross-Adaptation (CDCA): Align and fuse BEV features from vehicle and UAV using multi-scale correlation and attention-weighted BEV-space fusion, with empirical gains +4.8% mAP (Wang et al., 2024).
• Lift-Splat-Shoot (LSS): Project per-pixel features from RGB or LiDAR into BEV under calibrated intrinsics/extrinsics for unified spatial reasoning (Wang et al., 2024).

For multi-camera and multi-agent tracking, depth-based late aggregation raises HOTA (3D tracking metric) by +13 over 2D baselines, using clustering and orientation refinement on fused point clouds after track association (Le et al., 12 Sep 2025).

5. Benchmarks, Protocols, and Quantitative Evaluation

Cross-platform 3D perception research is grounded in rigorous benchmarking:

• 3EED: Tasks span single-platform in-domain, cross-platform zero-shot (train on vehicle, test on drone/quadruped), multi-object and multi-platform grounding. Accuracy@IoU and mIoU metrics reveal cross-domain gaps (vehicle-trained, Acc@25=52.4% on vehicle, 1.5% drone, 10.2% quadruped)—CPA, MSS, SAF reduce this discrepancy (Li et al., 3 Nov 2025).
• Pi3DET: Six adaptation scenarios with [email protected] for vehicle/drone/quadruped show Pi3DET-Net boosts BEV-mAP by +6.95 (drone) and +17.58 (quadruped), while ablations demonstrate complementary effects from RPJ, VPP, PFA, and GTD (Liang et al., 23 Jul 2025).
• VINet and PillarGrid: Demonstrate linear scaling in compute/bandwidth with system-level efficiency and state-of-the-art accuracy (VINet 41.5% BEV-mAP, 37.8% 3D-mAP, (Bai et al., 2022); PillarGrid 52.0% BEV-mAP, (Bai et al., 2022)).
• TranSimHub: Validates joint air–ground scenes via depth RMSE, segmentation mIoU, AP@IoU, and communication metrics. Explicit scenario coverage supports disaster response, infrastructure inspection, and cooperative autonomy (Wang et al., 17 Oct 2025).

6. Engineering Trade-offs, Deployment, and Limitations

Deployable architectures balance accuracy, compute, and communication cost:

• VINet reduces system-wide compute by 84% and comm cost by 94% compared to dense fusion, with LCUs performing light encoding and a CCU running the backbone (Bai et al., 2022).
• CMDT achieves real-time inference (<80 ms, 13.2 fps on RTX 3060) and is ROS-ready, making modular real-world deployment on resource-constrained robots feasible (He et al., 18 Apr 2025).
• PillarGrid operates at 2.5 MB/s comm bandwidth, suitable for V2I links, with latency ~50–100 ms on a modern GPU (Bai et al., 2022).
• HumanoidPano maintains geometric robustness under severe self-occlusion and motion drift, but future work is required for multi-LiDAR arrays, dynamic extrinsic calibration, temporal fusion, and expanding to outdoor domains (Zhang et al., 12 Mar 2025).

Limitations include vulnerability to sensor calibration errors, pseudo-label quality drift, incomplete adaptation for pure point-based backbones, and the need for manual hyperparameter tuning. The reality gap in simulation (e.g., weather, urban multipath) remains an active research area (Wang et al., 17 Oct 2025).

7. Open Problems and Future Directions

Areas for further development include:

• End-to-end dynamic calibration and viewpoint-invariant backbones beyond voxel or pillar methods (Zhang et al., 12 Mar 2025, Liang et al., 23 Jul 2025).
• Integration of temporal, dialogue, and multi-turn referring expressions, robust to adversarial instructions or environmental conditions (rain, nighttime) (Li et al., 3 Nov 2025).
• Multi-modal fusion expansion to encompass radar, event cameras, and thermal sensing.
• Automated adaptation of augmentation schedules and pseudo-label confidence thresholds based on cross-domain statistics (Feng et al., 13 Jan 2026).
• Richer unsupervised cross-modal alignment extending beyond 2D–3D and scene–instance discriminators (Zhang et al., 2023).

A plausible implication is that unified cross-platform 3D perception frameworks will underpin scalable collaborative autonomy and embodied intelligence in open-world, multi-agent environments. This convergence across modalities and domains is formalized by advances in dataset structure, normalization, feature adaptation, and modular open-source code.