BOLT: Online Lightweight Adaptation for Preparation-Free Heterogeneous Cooperative Perception

Abstract: Most existing heterogeneous cooperative perception methods depend on prior preparation like offline joint training or tailored collaborator-model adaptation. Such preprocessing is, however, generally impractical in real scenarios, as agents are usually independently trained by different developers and meet occasionally online. This work investigates \emph{preparation-free heterogeneous cooperative perception}, where agents use independently trained single-agent detectors without any pre-deployment coordination. We find direct cross-agent fusion under this setting greatly underperforms ego-only perception. We present BOLT, a lightweight plug-and-play module that adapts neighboring features online via ego-as-teacher distillation, requiring only ego predictions without ground-truth labels. BOLT leverages high-confidence ego perception features to guide cross-agent feature-domain alignment, while enabling neighbors to contribute features in the ego's low-confidence regions. With only 0.9M trainable parameters, BOLT improves AP@50 by up to 32.3 points over vanilla unadapted fusion in the preparation-free setting. It consistently outperforms ego-only results on DAIR-V2X and OPV2V, across different encoder pairs and fusion strategies. Code: https://github.com/sidiangongyuan/BOLT.

• The paper introduces BOLT, a lightweight plugin that enables online, label-free adaptation for heterogeneous cooperative perception without pre-deployment coordination.
• It employs adaptive instance normalization, a residual CNN adapter, and per-channel gating to align features between agents in an efficient plug-and-play manner.
• Empirical results demonstrate significant AP@50 improvements over naive fusion and ego-only baselines across diverse sensor modalities with minimal computational overhead.

Preparation-Free Heterogeneous Cooperative Perception with BOLT

Motivation and Problem Formulation

Distributed multi-agent perception is a cornerstone device for robust autonomous driving, but heterogeneous collaborative perception across vehicles and infrastructure is fundamentally challenged by independent model development, diverse sensor modalities, and incompatible feature spaces. Legacy solutions rely on extensive pre-deployment alignment—offline joint training, customized data, or collaborative protocol learning—which are difficult or infeasible at deployment in practical settings where agents are managed by different developers, updated asynchronously, and required to cooperate on the fly.

The paper formalizes the preparation-free heterogeneous cooperative perception setting, in which each agent uses an independently trained (single-agent) detector, with no coordinated multi-agent training or feature/interface alignment performed before deployment. Empirically, naive feature fusion in this regime notably degrades performance—sometimes below standalone perception—due to severe cross-agent distributional shifts and fusion module misinterpretation (Figure 1).

Figure 1: BOLT eliminates the need for pre-deployment cooperative training, introducing a lightweight plugin on the ego side for online, label-free adaptation, in contrast to conventional methods requiring restrictive preparation steps.

The BOLT Framework

The central contribution is BOLT, a lightweight, plug-and-play ego-side adaptation module that enables robust cooperative perception in the presence of heterogeneous collaborators without any pre-deployment coordination or label supervision. The plugin is injected at the interface where the ego agent receives a neighbor's feature (pre-fusion) and is the only component permitted to be updated online; all encoders, fusion modules, and detection heads remain frozen. BOLT's adaptation process is driven entirely by online ego-as-teacher distillation, exploiting the ego agent's own confident predictions as alignment supervision.

BOLT's architecture consists of three compositional operations:

1. Adaptive Instance Normalization (AdaIN): Quickly aligns per-channel feature statistics between the neighbor and ego agent, addressing first-order distribution mismatches.
2. Residual CNN Adapter: Provides lightweight, spatial-semantic transformation capacity via several residual blocks, initialized for identity to ensure stable adaptation from the first sample.
3. Per-Channel Gating: Learnable gating modulates the strength of adaptive correction channel-wise, supporting selective adaptation.

   Figure 2: BOLT’s architecture: the neighbor BEV feature is processed by the adaptive plugin (AdaIN → residual adapter → gating), then fused via a frozen module; only the plugin is updated via ego-as-teacher distillation.

Online Adaptation via Ego-as-Teacher Distillation

The distillation protocol leverages the ego’s own detection outputs, bifurcating the space of anchors into regions of high and moderate confidence:

• Preservation loss ($\mathcal{L}_{\mathrm{pres}}$): Where the ego is highly confident, the plugin is trained to replicate the ego predictions, ensuring that adaptation preserves strong egocentric detections and effectively aligns the feature domain.
• Enhancement loss ($\mathcal{L}_{\mathrm{enh}}$): In anchors with low-to-moderate ego confidence but nontrivial neighbor signal, the plugin is encouraged to raise confidence, allowing the neighbor to supply information in areas where the ego model is uncertain.

The learning proceeds strictly online: each sample is passed once (pre-update prediction is recorded for evaluation), and the plugin is updated only on its own current input. No ground-truth labels, external cooperative data, or multi-agent prealignment are used.

Empirical Evaluation

BOLT is evaluated on two established V2X collaborative perception benchmarks (DAIR-V2X and OPV2V) across both LiDAR and camera modalities, with diverse encoder pairings (e.g., PointPillars, SECOND, LSS-EfficientNet, LSS-ResNet-50). All models operate in the strictly preparation-free setting: each agent is trained independently, then plugged into a frozen fusion stack.

Key quantitative findings:

• Naive unadapted fusion consistently underperforms the ego-only baseline (e.g., dropping AP@50 by >30 points in the worst case).
• BOLT consistently converts degraded fusion into strong positive cooperation, yielding up to +32.3 AP@50 improvement over unadapted fusion, and always surpasses the ego-only baseline across datasets and encoder types.
• BOLT’s plugin remains highly parameter-efficient ($\sim$0.9M parameters)—two orders of magnitude smaller than standard detectors—and incurs only modest online inference update costs.

Qualitative BEV visualizations (Figure 3, Figure 4, Figure 5, Figure 6, Figure 7) confirm that BOLT recovers or enhances object detections missed in both the ego-only and naive-fusion cases.

Figure 3: BEV qualitative samples—BOLT’s plugin (right) robustly recovers vehicles missed by naive, mismatched fusion (left) on both real (DAIR-V2X) and synthetic (OPV2V) data.

BOLT's adaptation shows fast online convergence, typically achieving stable, superior detection performance with minimal drift across the stream (Figure 8).

Figure 8: BOLT achieves rapid and stable online AP@50 convergence, overtaking the ego-only baseline within a few samples.

Precision–recall analysis further indicates BOLT’s enhancement loss improves confident detection ranking without disproportionately increasing false positives (Figure 9).

Figure 9: Precision–recall curve: BOLT’s enhancement loss improves the high-confidence region and AP@50 relative to filtered distillation only.

Ablations, Comparisons, and Extensions

Comprehensive ablations probe the value of each architectural component and hyperparameter:

• Plugin components: The largest incremental benefit is attributed to the residual adapter; AdaIN/gating provide auxiliary gains.
• Plugin size: Substantially larger plugins yield only modest further AP improvements, indicating a favorable compute–accuracy trade-off for the default size.
• Enhancement loss: Essential for surpassing the ego-only baseline; aggressive thresholds or omitting enhancement reduces AP.

Comparisons with state-of-the-art heterogeneous cooperative methods (e.g., HEAL, STAMP, PHCP) confirm that BOLT surpasses both preparation-free and adapted versions of prior works when collaborative training is not permitted. Even under preparation-free deployment, HEAL and STAMP underperform, with STAMP notably collapsing to zero AP.

Evaluation further demonstrates robustness to stream ordering, multi-agent extensions (plugin per collaborator), and compatibility with various cooperative fusion backbones. BOLT’s adaptation overhead is dominated by plugin update time rather than forward latency or parameter footprint—well within typical deployment constraints.

Limitations and Future Directions

Limitations include the requirement of consistent BEV grid resolution between agents, dependence on the ego detector’s competence (weak-ego settings limit improvement), and a nonzero residual gap to the homogeneous, fully-collaborative upper bound. Cases where objects are visible only to the neighbor and undetected by the ego remain challenging, as supervision is absent for these regions. Additionally, while the plugin is agnostic to backbones, spatial misalignment (e.g., grid size mismatch) would necessitate further adaptation layers.

Future directions include lightweight adaptation of the fusion module, integration of cross-agent consistency constraints, development of spatial alignment modules for grid-mismatched encoders, and investigation of alternative self-supervised adaptation signals for objects outside the ego’s field of view.

Conclusion

This work establishes that online, label-free adaptation is a practical and effective paradigm for preparation-free heterogeneous cooperative perception. By introducing BOLT—a low-overhead, ego-side plugin trained via ego-as-teacher distillation—robust perception collaboration can be realized without any cooperative preparation, consistently outperforming the ego-only baseline and closing much of the heterogeneous fusion gap in open-world, real deployment scenarios. This lays new groundwork for multi-agent perception systems where explicit cross-agent coordination is not feasible, motivating ongoing research on scalable, fully autonomous interface repair in heterogeneous multi-agent AI.

---

Reference:

BOLT: Online Lightweight Adaptation for Preparation-Free Heterogeneous Cooperative Perception (2605.00405)