Class-Adaptive Cooperative Perception for Multi-Class LiDAR-based 3D Object Detection in V2X Systems

Abstract: Cooperative perception allows connected vehicles and roadside infrastructure to share sensor observations, creating a fused scene representation beyond the capability of any single platform. However, most cooperative 3D object detectors use a uniform fusion strategy for all object classes, which limits their ability to handle the different geometric structures and point-sampling patterns of small and large objects. This problem is further reinforced by narrow evaluation protocols that often emphasize a single dominant class or only a few cooperation settings, leaving robust multi-class detection across diverse vehicle-to-everything interactions insufficiently explored. To address this gap, we propose a class-adaptive cooperative perception architecture for multi-class 3D object detection from LiDAR data. The model integrates four components: multi-scale window attention with learned scale routing for spatially adaptive feature extraction, a class-specific fusion module that separates small and large objects into attentive fusion pathways, bird's-eye-view enhancement through parallel dilated convolution and channel recalibration for richer contextual representation, and class-balanced objective weighting to reduce bias toward frequent categories. Experiments on the V2X-Real benchmark cover vehicle-centric, infrastructure-centric, vehicle-to-vehicle, infrastructure-to-infrastructure, and vehicle-to-infrastructure settings under identical backbone and training configurations. The proposed method consistently improves mean detection performance over strong intermediate-fusion baselines, with the largest gains on trucks, clear improvements on pedestrians, and competitive results on cars. These results show that aligning feature extraction and fusion with class-dependent geometry and point density leads to more balanced cooperative perception in realistic vehicle-to-everything deployments.

• The paper introduces a novel class-adaptive cooperative perception architecture that dynamically adjusts feature aggregation for multi-class 3D object detection.
• It employs multi-scale window attention with dedicated fusion blocks and BEV enhancement to effectively handle variations in object scale and LiDAR density.
• The proposed method achieves superior mAP improvements, notably boosting detection performance for rare classes like pedestrians and trucks.

Class-Adaptive Cooperative Perception for Multi-Class 3D Object Detection in V2X Systems: An Expert Analysis

Introduction and Motivation

Cooperative perception via Vehicle-to-Everything (V2X) communication is central to overcoming the fundamental perceptual limitations of individual autonomous vehicles. Canonical approaches in cooperative 3D object detection typically employ class-agnostic fusion, homogenizing the treatment of distinct object classes—despite their disparate spatial scales, LiDAR return densities, and prevalence in data. This leads to persistent underperformance, especially for rare classes such as pedestrians and trucks. The presented paper introduces a class-adaptive cooperative perception architecture, instantiated specifically for multi-class LiDAR-based 3D object detection in V2X systems, and evaluated on the V2X-Real dataset across all major cooperation scenarios. The proposed framework fundamentally restructures feature aggregation and optimization objectives to account for inter-class heterogeneity, setting a new bar for balanced multi-class perception in realistic V2X deployments.

Methodological Innovations

The architecture introduces several synergistic contributions, each motivated by concrete empirical limitations of uniform-fusion paradigms:

1. Multi-Scale Window Attention with Class-Adaptive Routing: Expands the receptive field spectrum through pyramidally-structured local window sizes (2×2–16×16), and dynamically weights multi-scale feature responses via a learned router, conditioning on preliminary class predictions. This allows the network to allocate capacity to fine or coarse spatial context contingent on the geometric signature of the underlying object class.

   Figure 1: Schematic of the class-adaptive perception architecture, depicting multi-agent feature extraction, multi-scale window attention with class-specific routing, class-differentiated fusion, multi-scale BEV enhancement, and objective balancing.

2. Class-Specific Fusion Block: Unlike prior work, fusion is not monolithic; small and large object classes (e.g., pedestrians v. vehicles/trucks) traverse dedicated attentive branches with branch-specific parameterization—higher capacity and finer granularity for small objects, broader context for larger classes. The resulting outputs are concatenated and further refined, enabling representational specialization.
3. Multi-Scale BEV Enhancement via ASPP and SE Modules: Context aggregation is deepened using parallel atrous convolutions (dilations 1, 3, 6, 12) followed by squeeze-and-excitation. This mechanism provides multi-resolution context, further enhancing the architecture's ability to localize both small and large objects.
4. Class-Balanced Loss Optimization: The loss is explicitly class-reweighted based on effective sample counts, addressing training signal overdominance from majority classes and preventing margin collapse for rare classes. Weights reach up to 3× for pedestrian classification and 1.5–2× for trucks.

Experimental Setup and Baselines

Comprehensive evaluation is performed on the V2X-Real dataset, the current gold standard for real-world V2X cooperative perception, spanning five cooperation modalities: vehicle-centric, infrastructure-centric, vehicle-to-vehicle, infrastructure-to-infrastructure, and vehicle-to-infrastructure. Baselines for comparison include F-Cooper, AttFuse, and V2X-ViT, all reimplemented with multi-class extensions and standardized backbone architecture (PointPillars) to ensure fair ablation.

Figure 2: BEV visualization highlighting class imbalance and extreme scale variance: prominent vehicle presence, sparser pedestrians and trucks.

Empirical Results

The architecture demonstrates consistent [email protected] superiority (1.5–6.2 points absolute; see Table in source) across all cooperation patterns, with particularly notable margin improvement in underrepresented classes such as trucks (+6.7 points [email protected] in vehicle-centric mode compared to AttFuse). This is visually evident in comparative per-class AP and radar plots:

Figure 3: Per-class AP@50 averaged over cooperation modes, with most pronounced improvement in truck detection (+5.8 pp over baselines).

Figure 4: Radar plot in VC mode: class-adaptive method (red) encloses maximum area, driven by truck AP expansion (+6.7 pp over AttFuse).

Qualitative scenes substantiate reductions in both false positives and missed detections across modalities, particularly for pedestrians and trucks—a capability unobtainable by uniform-fusion approaches.

Figure 5: Qualitative comparison across VC, V2V, and IC modes: consistent improvement in localization accuracy and robustness, especially for rare classes.

Ablation and Analytical Insights

Ablation dissects the contribution of each system component. The greatest increment arises from the integration of multi-scale window attention with class-differentiated fusion, especially for trucks (+14.1 [email protected] in V2V mode post-M1 adoption). The multi-scale BEV enhancement (ASPP+SE) confers additional robustness at extreme range, evidenced by far-range AP increases. Incorporating the class-balanced loss consolidates improvements, stabilizing performance for underrepresented classes and yielding highest near-range mAP in infrastructure modes.

Figure 6: Ablation qualitative: Only the full model (with class-balanced loss) eliminates spurious pedestrian false positives and achieves calibrated predictions.

Range decomposition reveals that truck detection improvement is range-invariant—driven by structural architectural adaptation not point density. Pedestrian AP, while improved, remains fundamentally constrained by LiDAR sparsity, showing minimal sensitivity to spatial range.

Figure 7: Per-class AP@50 over BEV distance: class-adaptive variants sustain higher accuracy for vehicles and trucks at all distances; pedestrian detection shows limited range dependence.

Theoretical and Practical Implications

This work demonstrates that the uniform-fusion assumption is fundamentally suboptimal in the face of pronounced inter-class geometric and data-distributional heterogeneity. The class-adaptive paradigm is theoretically sound—allocating representational and computational resources as a function of detection difficulty and semantic context. Practically, its adoption facilitates more balanced detection pipelines for safety-critical but rare object classes, aligning with regulatory mandates for improved vulnerable road user (VRU) protection in autonomous vehicle deployments. Furthermore, each architectural block is modular, inviting transfer to other domains (e.g., long-tailed natural images, multi-sensor fusion in robotics).

The moderate increases in parameter count and inference cost, relative to the empirical gains, make the approach viable for deployment in real-world V2X systems, subject to bandwidth and hardware constraints.

Future Directions

Further research should investigate the co-design of communication-efficient class-adaptive fusion mechanisms for resource-constrained edge environments. Expanding to additional classes and heterogeneous sensor inputs (camera, radar) would enhance perceptual coverage. Continuous adaptation mechanisms leveraging self-supervised or federated learning paradigms at the edge could further improve robustness to distribution shift.

Conclusion

This paper provides compelling evidence that class-adaptive fusion, multi-scale context enhancement, and class-balanced optimization collectively resolve key limitations of class-agnostic cooperative perception in multi-agent, multi-class V2X scenarios. The architectural blueprint established herein offers a principled foundation for future research into perception for safety-critical, long-tailed detection tasks in distributed intelligent transportation systems.