BEVCon: Contrastive Learning for BEV Perception

• The paper introduces BEVCon, a contrastive framework that improves image-based BEV detectors by adding instance-aware supervision to both BEV and perspective-view features.
• It integrates two auxiliary losses—Instance Feature Contrast and Perspective Regional Contrast—with the standard detection loss to address weak supervision in the image backbone and view transformation modules.
• Empirical results on nuScenes demonstrate consistent performance gains across various BEV architectures without changing the underlying camera-to-BEV pipeline.

BEVCon is a contrastive learning framework for image-based Bird’s-Eye View perception that targets representation quality inside camera-based BEV detectors rather than redesigning the BEV encoder or the detection head. It augments a standard multi-camera 3D detection pipeline with two dense, instance-aware auxiliary objectives—an Instance Feature Contrast module in BEV space and a Perspective Regional Contrast module in perspective-view feature space—trained jointly with the baseline detection loss. In this formulation, the BEV representation is a spatial tensor $B \in \mathbb{R}^{H \times W \times C}$, and BEVCon treats both the image backbone and the view transformation module as under-supervised representation bottlenecks whose features can be improved by object-level contrastive structure (Leng et al., 6 Aug 2025).

1. Problem formulation and motivation

BEVCon addresses a specific weakness of image-based BEV perception: supervision reaching the image backbone and the image-to-BEV transformation is often weak, distorted, or overly indirect. In a typical BEV pipeline, multi-view images are processed by an image backbone, mapped into BEV space by a view transformation or BEV encoder, and then consumed by a 3D detection head. The paper argues that prior work has concentrated mainly on stronger BEV encoders, stronger task heads, depth estimation, temporal fusion, or larger pretrained backbones, while leaving representation learning inside BEV systems comparatively underexplored (Leng et al., 6 Aug 2025).

The two failure modes emphasized by BEVCon are architectural rather than dataset-specific. First, in dense-query systems such as BEVFormer, the detection loss must propagate through multiple transformer layers, BEV queries, and cross-attention operations before it reaches the image backbone. Second, in image-based BEV formation without explicit depth, features along a visual ray can remain ambiguous, leading to imprecise localization and “ray-shaped” artifacts. BEVCon therefore frames contrastive learning not as generic pretraining, but as a targeted way to impose additional structure on intermediate BEV and perspective-view representations.

The paper also tests more conventional contrastive strategies and treats their relative weakness as part of the motivation. MoCo-v2-style image-level pretraining on driving datasets and image-level contrast trained jointly with detection do not help much and can even hurt. The stated reasons are low sample diversity in driving data and a mismatch between global image-level contrast and the object-centric structure needed by autonomous-driving BEV models. This leads to the central BEVCon thesis: the useful level of contrast is not the whole image, but the object region in perspective space and the object instance in BEV space.

2. Training framework and integration into BEV detectors

BEVCon is inserted into an existing BEV detector rather than replacing it. During training, the model receives two augmented versions of the same multi-camera frame, denoted $(I, I')_t$. These are processed by a pair of EMA-updated image backbones, yielding paired multi-level perspective features $(P, P')_t$. A shared view transformation module then produces paired BEV representations $(B, B')_t$. The baseline detector and its 3D detection loss are kept intact, and the contrastive losses are simply added to the training objective (Leng et al., 6 Aug 2025).

The total loss is

$$\mathcal{L} = \lambda_{in}\mathcal{L}_{in} + \lambda_{pers}\mathcal{L}_{pers} + \mathcal{L}_{det}.$$

The implementation sets

$\lambda_{in}=1,\qquad \lambda_{pers}=1.$

This is a notable design choice: the auxiliary losses are not treated as small regularizers, but as equal-weight representation objectives trained alongside the detector.

The framework is explicitly intended to be architecture-agnostic. The paper evaluates it on depth-unprojection-based models, dense query-based models, and sparse query-based models. For BEVDet and BEVDet4D, the contrastive losses regularize depth-lifted BEV features. For BEVFormer, they act on dense BEV query outputs. For Sparse4D, the same principle is applied to sparse query embeddings instead of dense BEV grids. BEVCon therefore operates as a training geometry layered on top of existing image-based BEV families rather than as a new camera-to-BEV mechanism.

3. Instance Feature Contrast in BEV space

The BEV-space branch is the Instance Feature Contrast module. Its purpose is to make BEV features more discriminative and better localized by explicitly contrasting object-level BEV representations across two augmentations of the same frame. Given paired BEV features $B$ and $B'$ and BEV-space annotations $A=\{A_i\}_{i=1}^n$, the method extracts one feature per instance: $$f = \left\{ \mathcal{FP}(B, A_i)\mid i=1,\dots,n \right\}, \quad f' = \left\{ \mathcal{FP}(B', A_i)\mid i=1,\dots,n \right\}.$$ The positive pair is $(I, I')_t$0, meaning the same 3D object instance across two augmentations. Negative pairs are $(I, I')_t$1 for $(I, I')_t$2 (Leng et al., 6 Aug 2025).

For a positive pair, the per-instance loss is

$(I, I')_t$3

with normalized features so that the dot product is cosine similarity. The paper states that BEVCon uses RoI Align to extract instance features from low-resolution BEV feature maps. This is motivated by the mismatch between continuous annotation geometry and the coarse cell size of BEV grids, which can otherwise produce poor instance-feature alignment.

A further refinement is specific to dense-query BEV encoders such as BEVFormer. Instead of supervising only the final BEV output, BEVCon also contrasts intermediate BEV outputs from multiple transformer layers and combines them with exponential weighting parameterized by $(I, I')_t$4, set to $(I, I')_t$5 in implementation. This turns the method into a dense intermediate-supervision scheme for the BEV encoder stack rather than a purely terminal contrastive loss.

For sparse-query methods such as Sparse4D, the same idea is adapted to instance features extracted from sparse query embeddings rather than dense BEV rasters. This is important because it clarifies that BEVCon is not tied to one spatial representation; its invariant is the instance-level BEV-space contrast relation.

4. Perspective Regional Contrast and backbone supervision

The second branch, Perspective Regional Contrast, acts directly on perspective-view image features. Its motivation is that even strong BEV-space supervision still reaches the backbone only through the view transformation, which may weaken or distort gradients. BEVCon therefore introduces direct object-region supervision in perspective space using 2D annotations (Leng et al., 6 Aug 2025).

Let

$(I, I')_t$6

denote the multi-view, multi-level image features. For feature level $(I, I')_t$7, BEVCon pools per-object features

$(I, I')_t$8

and similarly from the augmented counterpart,

$(I, I')_t$9

Positive pairs are matching object regions from the two augmentations; negatives are different object regions.

A distinctive detail is scale-aware pooling. Because driving scenes contain heavy 2D box overlap under perspective projection and occlusion, pooling over the full box can include substantial content from nearby or partially occluded objects. BEVCon therefore shrinks each 2D box by a factor $(P, P')_t$0, using

$(P, P')_t$1

so that only a more central region contributes to the pooled feature. The ablations treat this as crucial: omitting scale-aware pooling weakens or destabilizes the full contrastive framework.

The perspective loss is applied at multiple backbone levels and then averaged. The paper does not provide a standalone closed-form equation for $(P, P')_t$2, but it describes it as the multi-level average of region-level contrastive losses over matched 2D object features. Conceptually, this branch gives the image backbone a direct BEV-relevant supervisory path that bypasses the view transformation bottleneck.

5. Empirical results and cross-architecture generalization

BEVCon is evaluated on nuScenes, using three major camera-BEV families: BEVDet, BEVDet4D, BEVFormer, and Sparse4D. The principal empirical claim is that the framework improves all of them without requiring extra labels or a redesigned detector (Leng et al., 6 Aug 2025).

The generalization table reported on the nuScenes validation set is summarized below.

Method	Baseline NDS / mAP	With BEVCon NDS / mAP
BEVDet (R50)	0.350 / 0.283	0.360 / 0.286
BEVDet4D (R50)	0.447 / 0.314	0.451 / 0.320
BEVFormer (R50)	0.354 / 0.252	0.375 / 0.276
Sparse4D (R101-DCN, single frame)	0.451 / 0.382	0.460 / 0.395
BEVFormer (ResNet-101-DCN)	0.517 / 0.415	0.528 / 0.424

The strongest relative gains in the table occur for the smaller BEVFormer configuration, where BEVCon improves NDS by $(P, P')_t$3 and mAP by $(P, P')_t$4. Sparse4D gains $(P, P')_t$5 NDS and $(P, P')_t$6 mAP. On the stronger BEVFormer model with ResNet-101-DCN, the gain is $(P, P')_t$7 NDS and $(P, P')_t$8 mAP, with improvements in mAOE, mAVE, and mAAE, although mATE is slightly worse numerically.

The ablations show that both contrastive branches matter. On BEVFormer-tiny with ResNet-50, the no-contrast baseline is $(P, P')_t$9 in NDS/mAP; Instance Contrast alone yields $(B, B')_t$0; Perspective Contrast alone yields $(B, B')_t$1; and the full framework yields $(B, B')_t$2. On BEVFormer-base with ResNet-101-DCN, the corresponding numbers are $(B, B')_t$3, $(B, B')_t$4, $(B, B')_t$5, and $(B, B')_t$6. A careful reading shows that the full framework does not always maximize every metric on the smaller model, but it does provide the best mAP and lower variance, while the larger model benefits most clearly from combining both branches.

A further comparison against standard contrastive learning is central to the paper’s interpretation. On BEVFormer-tiny, pretraining with MoCo v2 on nuScenes or ACO yields $(B, B')_t$7 or $(B, B')_t$8; image-level contrast plus detection gives $(B, B')_t$9; ImageNet contrastive pretraining gives $$\mathcal{L} = \lambda_{in}\mathcal{L}_{in} + \lambda_{pers}\mathcal{L}_{pers} + \mathcal{L}_{det}.$$0; and BEVCon reaches $$\mathcal{L} = \lambda_{in}\mathcal{L}_{in} + \lambda_{pers}\mathcal{L}_{pers} + \mathcal{L}_{det}.$$1. The paper uses this to argue that object-region and BEV-instance contrast are better aligned with BEV perception than naive global-image contrast.

6. Position within BEV representation learning

BEVCon belongs to a broader shift in BEV research from architecture-only changes toward explicit training-time supervision of intermediate representations. Earlier geometry-grounded work such as "BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud" formalized BEV construction as a two-stage semantics-plus-depth pipeline with analytic lifting and orthographic projection, emphasizing explicit geometry over learned image-to-BEV warping (Ng et al., 2020). By contrast, BEVCon leaves the projection mechanism unchanged and instead regularizes the learned features inside whatever BEV pipeline is already in use.

A second comparison is with "CLIP-BEVFormer: Enhancing Multi-View Image-Based BEV Detector with Ground Truth Flow" (Pan et al., 2024). That framework also adds training-only BEV-space supervision, but it does so by aligning object-level BEV crops with embeddings built from ground-truth labels and boxes and by injecting ground-truth object queries into the decoder. BEVCon differs in two respects that are explicit in the literature record: it supervises both BEV features and perspective-view backbone features, and it uses paired augmentations with annotation-defined positives rather than embeddings derived from a ground-truth encoder.

A third comparison is with "BEVContrast: Self-Supervision in BEV Space for Automotive Lidar Point Clouds" (Sautier et al., 2023). BEVContrast performs self-supervised contrastive learning on LiDAR BEV cell features aligned across temporally separated scans. BEVCon, by contrast, is image-based, joint with 3D detection, and annotation-driven. This distinction matters because BEVCon is sometimes informally grouped with self-supervised BEV contrastive learning; the paper’s own formulation does not support that interpretation.

This suggests that BEVCon is best understood not as a new BEV representation model, but as a detector-agnostic representation-learning layer for camera-based BEV perception. In that respect it is complementary to geometry-grounded pipelines, query-supervised BEV training schemes, and self-supervised LiDAR contrastive methods rather than a replacement for them.

7. Limitations, interpretation, and significance

Several limitations are explicit. The gains are larger on smaller or weaker baselines than on already strong models. Not every metric improves uniformly; for example, the full framework does not always maximize NDS in the small-model ablation, and mATE is slightly worse in the strong BEVFormer comparison. The method also depends on 2D and 3D object annotations to define regions and positive pairs, so it is not self-supervised in the strict sense (Leng et al., 6 Aug 2025).

There are also implementation sensitivities. The method requires augmentation policies that preserve correspondence between the same object across the two augmented views in both image space and BEV space. The ablations show that scale-aware pooling is crucial in the perspective branch and that multi-layer contrast matters in dense-query BEV encoders. The paper further notes that some formulation details remain implicit, including the exact form of the perspective contrast loss, the use of projection heads, and the details of EMA updates.

Two common misconceptions can therefore be addressed directly. First, BEVCon is not a new view transformer or a new BEV encoder; it is an auxiliary representation-learning framework layered on top of existing BEV detectors. Second, it is not a self-supervised pretraining method analogous to LiDAR-only BEV contrastive learning; its positives and negatives are defined using standard 2D and 3D training annotations.

Its significance lies in the claim that BEV perception is bottlenecked not only by camera geometry, depth estimation, temporal modeling, or decoder design, but also by the quality of the learned intermediate features. BEVCon’s central contribution is to show that dense, object-level contrast applied at the right locations—BEV instances and perspective-view regions—can improve BEV detectors across depth-lifted, dense-query, and sparse-query families without altering their output format or requiring additional data.