CaR1: Camera-Radar BEV Segmentation

• CaR1 is a camera-radar fusion architecture that transforms multi-view images and sparse radar point clouds into structured BEV features for vehicle segmentation.
• It employs a specialized pipeline with EfficientViT-L2 for camera encoding and a Point Transformer V3 for radar encoding to mitigate sensor disparities.
• Adaptive fusion balances contributions from both sensors dynamically, offering a cost-effective alternative to LiDAR-based systems in autonomous driving.

Searching arXiv for papers explicitly using the term "CaR1" and closely related nomenclature to disambiguate the topic. CaR1 is a camera-radar fusion architecture for bird’s-eye-view (BEV) vehicle segmentation in autonomous driving. It is presented as a multi-modal baseline built on BEVFusion, with two principal additions: a grid-wise radar encoding that transforms sparse radar point clouds into structured BEV features, and an adaptive fusion mechanism that dynamically balances camera and radar contributions in BEV space. In its main formulation, CaR1 takes multi-view camera images and automotive radar point clouds, aligns both modalities in a shared BEV representation, and predicts a BEV vehicle segmentation map; on nuScenes validation, it reports $57.6$ IoU for this task (Montiel-Marín et al., 12 Sep 2025).

1. Problem formulation and scope

CaR1 addresses BEV vehicle segmentation: predicting, on a top-down metric grid around the ego vehicle, which cells belong to vehicles. The paper formulates the output as a segmentation tensor

$$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$

where $|\mathcal{C}|$ is the number of classes and $H_{BEV}, W_{BEV}$ are the BEV grid dimensions. In the main setting, the task is binary vehicle segmentation, so the output class count is effectively $M=1$ in the segmentation head (Montiel-Marín et al., 12 Sep 2025).

The model is motivated by the complementary sensing properties of cameras and radar. Cameras provide strong semantic and appearance cues and dense spatial coverage in image space, but depth estimation is ill-posed and can be unreliable. Radar provides reliable distance and velocity and is robust under adverse weather, but is sparse, noisy, and has low angular resolution. CaR1 is therefore positioned as a cost-effective alternative to LiDAR-based systems, specifically in settings where one seeks stronger geometric cues than camera-only systems provide, without adopting LiDAR-centric hardware (Montiel-Marín et al., 12 Sep 2025).

The paper frames several specific challenges. First, there is view disparity: camera data are dense perspective images, whereas radar data are sparse unstructured point clouds. Second, BEV segmentation requires metric reasoning in 3D or ground-plane space, so camera depth ambiguity directly affects performance. Third, radar sparsity and low angular resolution make naive occupancy rasterization crude, while purely point-based radar processing does not align naturally with BEV fusion architectures. Fourth, fixed fusion is brittle when one sensor degrades because of weather, occlusion, blind spots, or limited radar returns. CaR1 is designed around a unified BEV space, grid-wise radar encoding, and adaptive fusion in order to address those constraints (Montiel-Marín et al., 12 Sep 2025).

2. Architectural lineage and end-to-end pipeline

CaR1 is iteratively built on BEVFusion rather than introduced as an entirely separate framework. The inherited baseline consists of a Swin Transformer Tiny plus FPN image encoder, a SECOND-based BEV decoder, a grid-sampling BEV segmentation head, and the PointPillars LiDAR encoder removed. CaR1 then modifies this baseline by increasing image resolution, replacing Swin-T with EfficientViT-L2, adding self-attention in the image encoder, introducing a radar encoder, replacing SECOND with Attention U-Net as the BEV decoder, and adding image and BEV augmentations (Montiel-Marín et al., 12 Sep 2025).

Its pipeline has five stages. Multi-view images are first encoded with EfficientViT-L2, FPN, and deformable attention; the resulting perspective features are projected into a voxelized 3D space and collapsed into BEV. Radar point clouds are encoded with a small Point Transformer V3, then scattered into a BEV grid using point $(x,y)$ locations, and subsequently processed by a pyramid aggregation network to add multi-scale context. The camera BEV and radar BEV features are then fused by an adaptive BEV fusion module. The fused representation is refined by an Attention U-Net. Finally, a segmentation head resamples the BEV features into metric coordinates and predicts the vehicle mask (Montiel-Marín et al., 12 Sep 2025).

This organization makes CaR1 a mid-level fusion system in BEV space. It is not early fusion at raw sensor level and not late fusion at final prediction level. Each modality is encoded separately into BEV, and the BEV tensors are fused only after modality-specific feature extraction. This choice is central to the method’s treatment of cross-modal alignment, because it preserves sensor-specific encoders while placing the actual fusion in a representation already suitable for road-scene reasoning (Montiel-Marín et al., 12 Sep 2025).

3. Camera branch, radar branch, and BEV alignment

The image branch uses EfficientViT-L2 as backbone. It extracts multi-scale perspective feature maps at $\times 8$, $\times 16$, and $\times 32$ downsampling. These are passed through an FPN producing feature maps with $C=256$ channels, followed by a Multi-Scale Deformable Attention module in a Deformable DETR style. The image-to-BEV transformation is geometric: centers of a predefined 3D voxel grid are projected onto image planes using camera intrinsics and extrinsics; valid projections are sampled by bilinear interpolation; features from overlapping camera views are averaged; and the resulting voxel-aligned features are reshaped into a BEV map with the vertical dimension collapsed into channels (Montiel-Marín et al., 12 Sep 2025).

The radar branch is the model’s most distinctive sensor-specific component. It uses a small Point Transformer V3 as a point-based encoder, with serialized attention over the ordered point cloud and a U-Net-shaped structure with transformer encoder and decoder. Learned point features are then projected to the BEV grid by computing each point’s $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$0 coordinates and aggregating via differentiable scatter addition. A pyramid aggregation network is applied afterward for multi-scale context. The paper emphasizes that this branch is best understood as point-based encoding followed by grid projection, rather than as a purely rasterized radar baseline (Montiel-Marín et al., 12 Sep 2025).

Several spatial parameters are explicitly specified. The evaluation BEV spans $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$1 meters in $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$2 and $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$3 meters in $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$4, corresponding to a $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$5 region centered on the ego vehicle. The output resolution is $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$6, producing a $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$7 grid. Internally, the network uses a $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$8 BEV grid at $$S \in \mathbb{R}^{|\mathcal{C}| \times H_{BEV} \times W_{BEV}}$$9 resolution and resamples after the segmentation head. For the image lifting stage, depth bins cover $|\mathcal{C}|$0 to $|\mathcal{C}|$1 with $|\mathcal{C}|$2 resolution. The radar input uses seven accumulated sweeps (Montiel-Marín et al., 12 Sep 2025).

The paper also identifies several under-specified elements. It states that the seven radar sweeps are accumulated and encoded using ordinal or one-hot representations, but it does not clearly enumerate the raw radar attributes used per point. It also does not specify the exact Point Transformer V3 variant depth or width, the output channel dimension before BEV scattering, or detailed temporal alignment formulas for sweep accumulation. These omissions are methodologically relevant because they limit exact reconstruction of the radar preprocessing stack from the paper alone (Montiel-Marín et al., 12 Sep 2025).

4. Adaptive fusion, decoding, and training objective

The fusion module operates on the camera BEV tensor $|\mathcal{C}|$3 and radar BEV tensor $|\mathcal{C}|$4. The paper gives the fusion equation as

$|\mathcal{C}|$5

where $|\mathcal{C}|$6 maps each modality to a common channel dimension, $|\mathcal{C}|$7 is an attention-based modality weight, and $|\mathcal{C}|$8 is a Squeeze-and-Excite channel recalibration module (Montiel-Marín et al., 12 Sep 2025).

Operationally, this fusion consists of per-modality channel alignment, adaptive weighting, weighted summation, and SE-based recalibration. The rationale given is that fixed fusion can fail when one modality is degraded: camera can suffer in poor visibility or occlusion, while radar can suffer from sparsity, clutter, and low angular resolution. The paper does not provide the explicit formula for $|\mathcal{C}|$9, however, and does not state whether the weights are scalar, per-channel, or spatially varying, nor whether they are normalized by softmax. The adaptive fusion mechanism is therefore conceptually clear but only partially specified mathematically (Montiel-Marín et al., 12 Sep 2025).

The BEV decoder is an Attention U-Net. Its attention gating is described by

$H_{BEV}, W_{BEV}$0

$H_{BEV}, W_{BEV}$1

where $H_{BEV}, W_{BEV}$2 is the previous decoder feature map, $H_{BEV}, W_{BEV}$3 is the corresponding encoder feature map from the skip connection, $H_{BEV}, W_{BEV}$4 is the attention mask, and $H_{BEV}, W_{BEV}$5 is sigmoid. The segmentation head then applies differentiable bilinear grid sampling to transform features from pixel space to metric BEV coordinates, followed by a double convolution block that maps from $H_{BEV}, W_{BEV}$6 channels to $H_{BEV}, W_{BEV}$7 classes; for vehicle segmentation, $H_{BEV}, W_{BEV}$8 (Montiel-Marín et al., 12 Sep 2025).

Ground truth is built by projecting all vehicle-like 3D bounding boxes into a unified vehicle category, making the label space effectively vehicle versus background. The training loss combines binary cross-entropy on the output head, BCE on an auxiliary head, and Dice loss on the output head: $H_{BEV}, W_{BEV}$9 with

$M=1$0

The paper does not give explicit formulas for BCE or Dice, but states that standard forms are implied (Montiel-Marín et al., 12 Sep 2025).

5. Optimization protocol and empirical results

Experiments are conducted on nuScenes, which is described as containing $M=1$1 scenes in total, split into $M=1$2 train, $M=1$3 validation, and $M=1$4 test scenes. The reported results are on the validation split. Training uses $M=1$5 epochs, $M=1$6 NVIDIA A100 $M=1$7GB GPUs, distributed data parallel, gradient accumulation, and an effective batch size of $M=1$8. The optimizer is AdamW with a one-cycle cosine annealing learning-rate schedule, $M=1$9 epoch warm-up, peak learning rate $(x,y)$0, and weight decay $(x,y)$1. The implementation uses PyTorch $(x,y)$2 and Lightning $(x,y)$3 (Montiel-Marín et al., 12 Sep 2025).

Input preprocessing and augmentation are also specified. Images are resized to $(x,y)$4. Image augmentations include random horizontal flip, zoom, and rotation. Radar uses seven accumulated sweeps and ordinal or one-hot representations. BEV augmentations applied to both inputs and labels include rotation, scaling, and flipping. Evaluation uses IoU or Jaccard index for BEV vehicle segmentation, with predictions thresholded at $(x,y)$5. Following prior work, vehicles with less than $(x,y)$6 visibility are excluded in evaluation, and inference uses batch size $(x,y)$7 (Montiel-Marín et al., 12 Sep 2025).

The main quantitative result is $(x,y)$8 IoU on nuScenes validation. In the comparison table reported in the paper, camera-only methods achieve $(x,y)$9 for FIERY Static, $\times 8$0 for BEVFormer, and $\times 8$1 for SimpleBEV. Camera-radar methods achieve $\times 8$2 for SimpleBEV++, $\times 8$3 for SimpleBEV (C+R), $\times 8$4 for BEVCar, $\times 8$5 for CRN, $\times 8$6 for BEVGuide, and $\times 8$7 for CaR1. The paper therefore supports two simultaneous statements: CaR1 clearly outperforms the listed camera-only methods, and it is competitive with state-of-the-art camera-radar methods, but it is not the top entry in the reported table (Montiel-Marín et al., 12 Sep 2025).

This comparative position matters because the phrase “on par with state-of-the-art” can be misunderstood as “best reported result.” The numbers in the paper do not support that stronger claim. They support the narrower conclusion that CaR1 is close to leading methods while remaining architecturally simpler and explicitly framed as a baseline (Montiel-Marín et al., 12 Sep 2025).

6. Ablations, interpretation, and limitations

The ablation study shows how the final model is assembled. The starting baseline achieves $\times 8$8 IoU. Higher image resolution adds $\times 8$9, yielding $\times 16$0. Replacing Swin-T with EfficientViT-L2 adds $\times 16$1, yielding $\times 16$2. Adding image encoder self-attention adds $\times 16$3, yielding $\times 16$4. Adding the PTv3 plus convolutional radar encoder adds $\times 16$5, yielding $\times 16$6. Replacing SECOND with Attention U-Net adds $\times 16$7, yielding $\times 16$8. Adding data augmentation contributes $\times 16$9, reaching the final $\times 32$0 IoU (Montiel-Marín et al., 12 Sep 2025).

These ablations indicate that the largest gains come from three sources: a stronger image encoder, introduction of the radar branch, and replacement of the BEV decoder with Attention U-Net. This supports the interpretation that CaR1’s performance is not attributable to fusion alone; it depends on improvements at multiple stages of the BEV pipeline. The results also suggest that radar contributes substantially to final segmentation quality, especially relative to camera-only baselines (Montiel-Marín et al., 12 Sep 2025).

The paper includes an extensive account of what remains unspecified. It does not provide isolated ablations for the exact grid-wise radar encoding against alternative radar encodings, for adaptive fusion weights versus a simple sum or concatenation, for ordinal versus one-hot sweep encoding, or for SE recalibration versus no SE. It likewise omits the exact radar point feature vector, the exact adaptive weighting network, the exact Point Transformer V3 configuration, and the architecture of the auxiliary head. These are not minor editorial gaps: they delimit the difference between a conceptually reproducible design and a fully specified implementation (Montiel-Marín et al., 12 Sep 2025).

The paper also delimits the scope of demonstrated generalization. CaR1 is evaluated only on vehicle segmentation. A plausible implication is that the fused BEV representation could support tasks such as multi-class semantic segmentation, occupancy prediction, 3D object detection, tracking, or motion forecasting, because the architecture produces a fused BEV tensor. The paper, however, does not report such extensions, and therefore does not establish them empirically (Montiel-Marín et al., 12 Sep 2025).