Diffusion on Reinforced Cooperative Perception

• The paper introduces DRCP, a cooperative perception framework that couples cross-modal and cross-agent fusion (PPXX) with a one-step diffusion refinement module (MDMA) to enhance BEV features.
• It employs innovative grid sector sampling and occupancy-guided pyramid fusion to adaptively refine features, mitigating partial detections and noise accumulation.
• Empirical results show significant AP improvements on DAIR-V2X and OPV2V, demonstrating robust performance under noisy, real-world conditions.

Diffusion on Reinforced Cooperative Perception (DRCP) denotes a cooperative perception framework for autonomous vehicles and related mobile robotic platforms in which multi-agent, multi-modal fusion is followed by a lightweight diffusion-based refinement stage in bird’s-eye-view (BEV) space. In the explicit usage of the acronym, DRCP is introduced as a real-time deployable system for Vehicle-to-Everything communication that targets partial detections, noise accumulation, and task-misaligned BEV features through two coupled modules: Precise-Pyramid-Cross-Modality-Cross-Agent (PPXX) and Mask-Diffusion-Mask-Aggregation (MDMA) (Li et al., 29 Sep 2025). In the surrounding literature, diffusion is also used for cooperative denoising, conditional latent fusion, and corruption-robust supervision, which places DRCP within a broader diffusion-augmented cooperative perception lineage that includes CoDiff and CoopDiff (Huang et al., 17 Feb 2025, Chen et al., 2 Mar 2026).

1. Definition and problem setting

DRCP is formulated for cooperative perception scenarios in which vehicles or infrastructure nodes exchange sensor-derived representations to enlarge coverage and improve 3D detection. The framework is motivated by three failure modes identified in multi-agent perception: partial detections or incomplete observations, noise accumulation across agents and modalities, and BEV features that remain off the “task-optimal manifold” even after fusion (Li et al., 29 Sep 2025). The practical claim is that cooperative perception should not terminate at feature aggregation; the fused BEV representation should be reinforced and refined so that downstream detection becomes more stable under hard cases.

The target difficulty is not only missing viewpoint coverage. The cited problem formulation emphasizes that cross-agent alignment errors, sensor noise, and camera-to-BEV depth estimation errors can accumulate, especially in heterogeneous LiDAR-camera systems. In that setting, weak or ambiguous BEV activations can propagate through the fusion stack and degrade box localization and orientation estimates. DRCP addresses this by coupling a geometry-aware cooperative fusion backbone with a task-oriented one-step denoiser that modifies the fused BEV only where the model predicts that correction is useful (Li et al., 29 Sep 2025).

Within the literature, this places DRCP adjacent to diffusion-based cooperative perception but distinct in scope. CoDiff reframes collaborative 3D object detection as conditional denoising in a latent space under pose and delay noise (Huang et al., 17 Feb 2025), whereas CoopDiff addresses multi-agent robustness under environmental and sensor-level corruptions through a teacher-student denoising framework (Chen et al., 2 Mar 2026). DRCP, by contrast, is centered on post-fusion reinforcement of BEV features under real-time deployment constraints.

2. PPXX: cross-modal and cross-agent fusion backbone

The first component of DRCP is PPXX, a cross-modal and cross-agent fusion module with three stages: camera-intrinsics-aware cross-modal fusion, integrated pyramid cross-agent fusion, and adaptive convolution refinement at the final BEV (Li et al., 29 Sep 2025). Its purpose is to produce a strong fused BEV before diffusion refinement is applied.

For a given agent, PPXX begins with LiDAR BEV features $F^{BEV} \in \mathbb{R}^{C_1 \times H_1 \times W_1}$ and camera features $$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$. The camera pose is transformed into the BEV frame, and the image width is partitioned column-wise into angular sub-sectors using the camera intrinsics:

$$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$

$$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$

BEV sampling is then performed radially as

$r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$

$$x_{m,n} = p^{BEV}_{i_{\text{cam}},x} + r_n \cos\theta_m, \qquad y_{m,n} = p^{BEV}_{i_{\text{cam}},y} - r_n \sin\theta_m,$$

yielding a sub-BEV feature map $$F^{sub\_BEV} \in \mathbb{R}^{C_1 \times H_1 \times W_2}$$. This “Grid Sector Sampling” is presented as preserving angular continuity and improving image-to-BEV correspondence relative to coarser projection rules.

Cross-modal fusion is performed by Intrin-RG-Attn. After reshaping, positional embedding, and channel alignment, multi-head attention is applied column-wise with LiDAR BEV features as query and camera features as key and value. The resulting fused representation is written as

$$F^{Fused\_BEV} = \text{Intrin-RG-Attn}(F^{BEV}, F^{i_{\text{cam}}}).$$

A central argument in the DRCP paper is that this attention-based design enriches LiDAR BEV semantics without relying on explicit camera depth prediction, which the paper identifies as a source of degradation in some prior LiDAR-camera cooperative systems.

Cross-agent fusion is carried out over a three-scale pyramid $\{F^{Fused\_BEV}_{k,(s)}\}_{s=1}^{3}$ for each agent $k$. At each scale, an occupancy head predicts $$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$0 using sigmoid focal loss. These occupancy maps define agent weights

$$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$1

and the scale-specific fused feature is

$$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$2

followed by pyramid concatenation

$$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$3

This occupancy-guided design emphasizes informative regions and agents rather than treating all contributions uniformly.

PPXX concludes with adaptive multi-scale convolution. Given $$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$4, three convolutions with kernel sizes $$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$5, $$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$6, and $$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$7 produce $$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$8, $$F^{i_{\text{cam}}} \in \mathbb{R}^{C_2 \times H_2 \times W_2}$$9, and $$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$0. A dynamic $$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$1 convolution followed by softmax yields spatially varying weights $$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$2 satisfying $$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$3, and the final PPXX output is

$$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$4

The stated role of this block is to adapt receptive field size per location for small versus large objects and for near-versus-far interactions across agents.

3. MDMA: one-step diffusion refinement in BEV space

The second component of DRCP is MDMA, a lightweight diffusion-based refinement module that operates directly on BEV features and is designed for hard detection cases (Li et al., 29 Sep 2025). Its guiding premise is that the fused BEV is often noisy but already informative; therefore the objective is not full generative reconstruction, but selective task-oriented correction.

MDMA accepts an origin feature map $$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$5 and executes four stages: seed condition extraction, forward perturbation, single-step conditioned denoising, and residual fusion with a learned mask. In seed extraction, a channel-wise confidence mask is predicted as

$$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$6

and the seed condition is

$$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$7

The paper describes this mask as retaining reliable channels and suppressing unstable ones so that conditioning remains numerically stable.

The forward perturbation follows the familiar Gaussian noising rule

$$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$8

However, the reverse process does not use a standard iterative DDPM chain. Instead, MDMA employs a one-step deterministic mapping,

$$p_{\text{width}} = \frac{W_{\text{camera resolution}}}{W_2}, \qquad p_{\text{origin},m} = m \cdot p_{\text{width}} + \frac{p_{\text{width}}}{2},$$9

where $$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$0 is a compact U-Net with 2 downsampling blocks and 2 upsampling blocks. The paper characterizes this as a DDIM-style collapse to a single step for real-time use.

Residual fusion is controlled by a second mask,

$$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$1

with final refined output

$$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$2

or equivalently

$$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$3

The two-mask design is central to MDMA. The reported ablation states that “no mask” is disastrous, “mask 1 only” is insufficient, “mask 2 only” helps but is weaker, and “both masks together” work best.

Conceptually, MDMA is presented as aligning BEV features closer to the task-optimal manifold. The refinement is explicitly described as operating in a high-SNR, low-variance regime: it does not redraw the entire BEV but learns small residual corrections that suppress ambiguous responses, strengthen useful cues, and enlarge object footprints in a semantically coherent way. This is the sense in which the paper uses “reinforced” cooperative perception.

4. Detection head, optimization, and deployment path

DRCP uses an anchor-based detection head on a final BEV feature of size $$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$4, with 6 anchors per spatial location (Li et al., 29 Sep 2025). The classification head is $$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$5, the regression head is $$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$6, the direction head is $$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$7, and the occupancy head is $$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$8. Bounding boxes are parameterized as $$\Delta p_m = \frac{p_{\text{origin},m}-c_x}{f_x}, \qquad \theta_m = \arctan(\Delta p_m) + \theta^{BEV}_{i_{\text{cam}}}.$$9. At inference time, classification logits are converted to probabilities, regression and direction outputs are combined, and non-maximum suppression is applied.

The total loss is

$r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$0

Here $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$1 and $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$2 are sigmoid focal loss with $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$3 and $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$4, with $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$5; $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$6 is weighted smooth $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$7 with $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$8 and $r_n = \frac{n}{H_1}R, \qquad R=\frac{W_1}{2},$9; and $$x_{m,n} = p^{BEV}_{i_{\text{cam}},x} + r_n \cos\theta_m, \qquad y_{m,n} = p^{BEV}_{i_{\text{cam}},y} - r_n \sin\theta_m,$$0 is softmax cross-entropy with $$x_{m,n} = p^{BEV}_{i_{\text{cam}},x} + r_n \cos\theta_m, \qquad y_{m,n} = p^{BEV}_{i_{\text{cam}},y} - r_n \sin\theta_m,$$1. The optimization scheme is staged. PPXX-only is trained end-to-end, while MDMA is introduced through a two-stage process in which a non-diffusion baseline is trained first and then fine-tuned with the diffusion module for BEV enhancement.

The implementation uses Adam, with initial learning rate $$x_{m,n} = p^{BEV}_{i_{\text{cam}},x} + r_n \cos\theta_m, \qquad y_{m,n} = p^{BEV}_{i_{\text{cam}},y} - r_n \sin\theta_m,$$2 for the baseline, decay by $$x_{m,n} = p^{BEV}_{i_{\text{cam}},x} + r_n \cos\theta_m, \qquad y_{m,n} = p^{BEV}_{i_{\text{cam}},y} - r_n \sin\theta_m,$$3 at epoch 27 on DAIR-V2X and epoch 38 on OPV2V, and MDMA fine-tuning learning rate $$x_{m,n} = p^{BEV}_{i_{\text{cam}},x} + r_n \cos\theta_m, \qquad y_{m,n} = p^{BEV}_{i_{\text{cam}},y} - r_n \sin\theta_m,$$4 for up to 3 epochs. For fairness, LiDAR is encoded with PointPillar and cameras with the first five layers of ResNet101; PPXX is built on HEAL by replacing BEV-related modules with PPXX and attaching MDMA.

The deployment profile is a major part of the DRCP claim. The full system runs at 68 ms/frame end-to-end, compared with 54 ms/frame for PPXX only. Component timing is reported as about 9.5 ms for Intrin-RG-Attn, less than 1 ms for Adaptive Convolution, and about 15 ms for MDMA. Training uses a single NVIDIA RTX 6000 Ada, and inference is reported on an NVIDIA RTX 3060. The comparison table gives 44.1M / 38.5M parameters for two DRCP configurations. The paper also reports that a lightweight autoencoder for sensory sharing can achieve up to 32× compression with only 0.8% accuracy loss and an additional 2 ms runtime.

5. Empirical performance and robustness profile

The empirical evaluation is conducted on DAIR-V2X and OPV2V (Li et al., 29 Sep 2025). DAIR-V2X is described as a real-world dataset from the Beijing Autonomous Driving Zone with about 9K frames, vehicle-plus-RSU sensing, LiDAR, and 1920×1080 cameras; OPV2V is a simulated CARLA-based dataset with over 11K frames, 2–7 vehicles per frame, 64-channel 360° LiDAR, and four 800×600 cameras per vehicle.

Setting	Metric	Value
DAIR-V2X	AP30 / AP50 / AP70	0.878 / 0.836 / 0.674
OPV2V	AP30 / AP50 / AP70	0.978 / 0.976 / 0.951
Full system	End-to-end runtime	68 ms/frame

On DAIR-V2X, the reported improvement over the previous best is +4.6% AP30, +4.6% AP50, and +5.1% AP70. On OPV2V, the reported improvement is +1.0% AP30, +1.3% AP50, and +2.5% AP70. The paper explicitly attributes the larger DAIR-V2X gains to the fact that it is noisier and more realistic, leaving more room for the method to help.

The ablation studies assign distinct roles to the PPXX and MDMA components. On DAIR-V2X, the baseline scores 0.860 / 0.815 / 0.637 at AP30 / AP50 / AP70, adding Intrin-RG-Attn gives 0.867 / 0.822 / 0.652, adding Adaptive Convolution gives 0.869 / 0.824 / 0.651, and full PPXX reaches 0.873 / 0.831 / 0.668. For MDMA, the ablation reports 0.694 / 0.609 / 0.215 with no mask, 0.833 / 0.747 / 0.407 with mask 1 only, 0.867 / 0.821 / 0.642 with mask 2 only, and 0.879 / 0.833 / 0.658 with full MDMA. These results are used to argue that the first mask stabilizes conditioning, the second controls useful residual fusion, and both are necessary.

The robustness evaluation covers pose noise, agent count, and communication compression. Under Gaussian perturbations to pose data in DAIR-V2X, DRCP is reported to retain its advantage across noise levels, supporting the claim that fusion remains stable under localization error. On OPV2V, more agents improve performance, but the gain diminishes as the scene becomes cleaner and more complete; with fewer agents, MDMA is said to be especially helpful because it compensates for missing or partial perception. The paper also highlights a negative result for prior multi-modal methods: HEAL and CoBEVT degrade under LiDAR-camera configurations when explicit camera depth estimation corrupts fusion, with reported drops of DAIR-V2X AP30: 0.588 (-19.9%) / 0.776 (-5.6%) and OPV2V AP50: 0.643 (-29.2%) / 0.854 (-10.9%).

6. Relation to adjacent diffusion-based cooperative perception

DRCP sits within a rapidly developing cluster of diffusion-based cooperative perception methods, but the role assigned to diffusion differs substantially across papers. In CoDiff, diffusion replaces the middle-stage aggregation operator in collaborative 3D object detection. The pipeline is per-agent feature extraction, autoencoder-based perceptual compression, and conditional latent diffusion, with the fused representation generated as

$$x_{m,n} = p^{BEV}_{i_{\text{cam}},x} + r_n \cos\theta_m, \qquad y_{m,n} = p^{BEV}_{i_{\text{cam}},y} - r_n \sin\theta_m,$$5

The model uses a pre-trained autoencoder, compresses to an 8-channel latent space with best compression ratio 32×, injects conditions by concatenation, and uses DDPM with 8 sampling steps by default. It is presented as the first work to apply diffusion models to multi-agent collaborative perception and is evaluated mainly against pose noise and time-delay noise rather than post-fusion BEV reinforcement (Huang et al., 17 Feb 2025).

CoopDiff addresses a different robustness axis: corruption under multi-agent cooperative perception. It adopts a teacher-student paradigm in which a Quality-Aware Teacher performs voxel-level early fusion with Quality of Interest weighting and semantic guidance, then produces clean supervision features through a diffusion denoiser. A Dual-Branch Diffusion Student reconstructs the teacher’s clean target under corruption and decodes with Ego-Guided Cross-Attention. The framework is evaluated on OPV2Vn and DAIR-V2Xn, each with six corruption types—beam missing, motion blur, fog, EMI, water, and echo—and reports clean-data scores of 0.9053 / 0.8357 on OPV2Vn and 0.8069 / 0.6644 on DAIR-V2Xn, with the lowest mRCE of 12.94% on OPV2V and 26.79% on DAIR-V2X. Its diffusion component is thus a teacher-generated clean supervision mechanism plus a student-side denoising task, not the one-step residual BEV corrector used in DRCP (Chen et al., 2 Mar 2026).

A broader, looser use of the idea appears in the multi-drone framework that combines federated diffusion and NeRF. That system uses semantics-and-pose communication, target-side diffusion-based view synthesis, and local NeRF updates, with an optional future-work reinforcement-learning layer called PA-MACRPO that models communication and resource allocation as a DEC-POMDP with reward

$$x_{m,n} = p^{BEV}_{i_{\text{cam}},x} + r_n \cos\theta_m, \qquad y_{m,n} = p^{BEV}_{i_{\text{cam}},y} - r_n \sin\theta_m,$$6

The paper explicitly states that it does not define “Diffusion on Reinforced Cooperative Perception” as a formal algorithm, but can be regarded as DRCP-adjacent because it combines cooperative perception, diffusion as a shared generative prior, and a proposed reinforcement-learning extension for adaptive communication (Pourmandi, 1 Aug 2025).

A common misconception is therefore to equate DRCP with reinforcement learning. In the explicit DRCP framework, “reinforced cooperative perception” refers to BEV feature reinforcement by MDMA after PPXX fusion, not to an end-to-end policy-learning controller (Li et al., 29 Sep 2025). By contrast, reinforcement learning appears only as an extension in the federated diffusion–NeRF proposal (Pourmandi, 1 Aug 2025). Another misconception is that diffusion always replaces fusion. CoDiff uses diffusion as the fusion engine, CoopDiff uses it to construct clean supervision and denoise corrupted cooperative features, and DRCP uses it as a compact post-fusion residual corrector.

7. Limitations and open directions

The DRCP paper notes or implies several limitations. First, the reported 4 MB sensory-sharing budget may be high for real-world V2X deployment (Li et al., 29 Sep 2025). Second, MDMA is validated primarily for BEV detection rather than for broader cooperative perception tasks. Third, the single-step diffusion design remains partly heuristic: the paper reports comparisons between DDIM and DDPM schedules and states that performance depends on the chosen number of noise steps. Fourth, the future-work directions named in the paper are temporal modeling, dynamic information sharing, and broader validation on other datasets such as NuScenes for single-agent MDMA effects.

The neighboring literature exposes additional unresolved design choices. CoDiff reduces diffusion cost through latent compression, but still relies on a two-stage training pipeline and iterative sampling, and its robustness study centers on synthetic pose and delay noise rather than the broader corruption spectrum considered in CoopDiff (Huang et al., 17 Feb 2025, Chen et al., 2 Mar 2026). The federated diffusion–NeRF proposal raises a different class of open questions—communication coding, non-IID federated convergence, and the absence of an implemented end-to-end MARL controller—because its reinforcement component is explicitly aspirational rather than core functionality (Pourmandi, 1 Aug 2025).

Taken together, these works suggest that DRCP is most precisely understood in two layers. In the narrow sense, it is the PPXX-plus-MDMA framework of “DRCP: Diffusion on Reinforced Cooperative Perception for Perceiving Beyond Limits” (Li et al., 29 Sep 2025). In a broader, literature-level sense, it denotes a research direction in which cooperative perception is strengthened by diffusion-based denoising, feature-space refinement, or generative completion, while questions of communication efficiency, temporal consistency, and adaptive multi-agent control remain active topics rather than settled components.