CenterRadarNet: 3D Radar Perception

• The paper presents a joint framework that leverages raw 4D FMCW radar tensors for full 3D detection and re-identification-based tracking, achieving state-of-the-art performance on K-Radar.
• CenterRadarNet employs an HR3D backbone with multi-resolution feature fusion and a CenterPoint-style detection head to convert volumetric inputs into precise BEV heatmaps and 3D bounding boxes.
• The framework integrates appearance embeddings via JDE training and a Kalman filter-based online tracker to maintain identity consistency over time, enhancing detection robustness under adverse conditions.

Searching arXiv for CenterRadarNet and closely related radar perception papers. CenterRadarNet is a joint 3D object detection and multi-object tracking framework for automotive perception that operates directly on raw 4D FMCW radar tensors and exploits the full Doppler–range–azimuth–elevation signal structure rather than pre-processed radar point clouds (Cheng et al., 2023). It is formulated as a single-stage, anchor-free detector that predicts dense bird’s-eye-view (BEV) object-center heatmaps, full 3D bounding-box attributes, and a discriminative appearance embedding for each detected object, with an online tracker layered on top for re-identification (re-ID) and identity-consistent association over time. On K-Radar, it is reported to achieve the state of the art for radar-only 3D detection and to provide the first 3D radar-based tracking result on K-Radar V2 (Cheng et al., 2023).

1. Problem setting and motivation

CenterRadarNet is situated in the broader effort to make autonomous and assisted driving perception robust under adverse weather and illumination. Automotive FMCW radar in the 77–81 GHz band provides long-range sensing, direct Doppler measurement, and relative robustness to rain, snow, fog, and poor lighting, but it is also characterized by lower angular resolution and weaker semantic cues than cameras or LiDAR (Cheng et al., 2023).

The framework addresses limitations identified in prior radar-based 3D perception. Earlier RF-tensor methods commonly used 3D tensors in Doppler–range–azimuth space, which allowed prediction of location, heading, and footprint size in BEV, but did not fully exploit elevation for full 3D box recovery. Other approaches relied on CFAR and clustering to convert radar returns into point clouds, introducing sparsity and discarding raw spatiotemporal structure. Many methods also treated detection in isolation and did not learn appearance embeddings or object identities for multi-object tracking. CenterRadarNet was introduced to overcome these constraints by combining full 4D radar input, full 3D box regression, and JDE-style re-identification within a unified radar-only pipeline (Cheng et al., 2023).

2. Radar representation and data processing

The input is a 4D FMCW radar tensor

$T \in \mathbb{R}^{D \times Z \times Y \times X},$

where $D$ denotes Doppler bins, $Z$ elevation, $Y$ lateral extent, and $X$ longitudinal range. Each cell stores a power or intensity value after FFT-based processing, corresponding to radar backscatter in a small voxel of 3D space (Cheng et al., 2023).

The tensors are provided in polar coordinates and converted to Cartesian coordinates. CenterRadarNet uses a fixed voxel size of $$0.4\,\text{m} \times 0.4\,\text{m} \times 0.4\,\text{m}$$, with detection range

$$(x_{\min}, x_{\max}, y_{\min}, y_{\max}, z_{\min}, z_{\max}) = (0, 72, -15, 15, -2, 7.6)\,\text{m}.$$

No hand-crafted CFAR or thresholding is applied before the network; the raw tensor is used directly for learning. CFAR is retained only for visibility analysis and ground-truth weighting during training (Cheng et al., 2023).

Temporal information is not modeled with explicit recurrent or temporal layers. Instead, the network is a per-frame feedforward 3D CNN, while temporal consistency is introduced during training through JDE supervision. Mini-batches contain pairs of frames $t_1$ and $t_2$ with $|t_1-t_2|<T$ and $D$0, and cross-frame triplet loss is applied to appearance embeddings to make them identity-consistent over short time intervals. Doppler is treated as an input axis encoding instantaneous motion (Cheng et al., 2023).

3. Architecture

CenterRadarNet comprises three principal subsystems: an HR3D backbone for volumetric feature extraction, CenterPoint-style BEV heads for detection and embedding, and an online tracker for association (Cheng et al., 2023).

The HR3D backbone is a fully 3D HRNet-style network. The Doppler axis is treated as channels of a 3D CNN, after which the model maintains multiple parallel branches at different spatial resolutions. From stage to stage, each new branch has half the spatial resolution in each dimension and twice the number of channels. Within a stage, branches perform 3D convolutions independently and then exchange information through 3D downsampling and upsampling. This preserves a high-resolution branch while aggregating broader contextual semantics from lower-resolution branches. The experimental configuration is $D$1, meaning three stages, one module per stage, and two convolution blocks per branch within a module (Cheng et al., 2023).

After HR3D, a high-resolution 3D feature map

$D$2

is converted for BEV reasoning by folding the elevation axis into channels: $D$3 Subsequent 2D convolutions produce a BEV feature map used by all output heads. This encoding is critical because it allows height and vertical center information to remain accessible even though prediction occurs in BEV (Cheng et al., 2023).

The head structure is visibility- and identity-aware. A shared BEV feature map feeds parallel convolutional heads for class-wise center heatmaps, 3D box regression, and appearance embedding. Because HR3D already performs multi-scale fusion, no additional FPN is introduced (Cheng et al., 2023).

4. Detection formulation and joint embedding

Detection follows a CenterNet/CenterPoint-style formulation. For each class, the network predicts a dense center heatmap in BEV together with regression outputs for center offsets, dimensions $D$4, orientation, and vertical position. Yaw is parameterized as $D$5 and $D$6, with decoding by

$D$7

At inference, center peaks are selected from the predicted heatmap and combined with the regression outputs to reconstruct 3D boxes (Cheng et al., 2023).

A distinctive component is the radar visibility analysis used to soften ground-truth heatmaps. The method applies 3D CA-CFAR over the radar tensor, counts the number of CFAR points inside each ground-truth 3D box, and uses that count as a proxy for radar detectability. The target heatmap for an object $D$8 centered at $D$9 is

$Z$0

with $Z$1 and $Z$2, and $Z$3 determined by the CFAR count. Objects with zero CFAR points are filtered from training to stabilize learning. The classification loss is the modified focal loss inherited from CenterNet/CenterPoint (Cheng et al., 2023).

The appearance branch predicts a per-pixel embedding vector

$Z$4

with embedding dimension $Z$5. Training uses a triplet hard loss with cosine-similarity-based distance so that embeddings of the same physical object across nearby frames are closer than embeddings of different objects by at least a margin $Z$6. The multi-task objective is

$Z$7

This JDE formulation couples detection and re-ID in a single forward pass rather than treating identity modeling as a separate downstream stage (Cheng et al., 2023).

5. Online tracking formulation

The online tracker is adapted from BoT-SORT/ByteTrack principles and combines motion and appearance cues. Motion is modeled with a 3D Kalman filter using a constant-velocity state vector of dimension 16: $Z$8 The state evolves according to

$Z$9

where $Y$0 is derived from current-frame detections (Cheng et al., 2023).

Association uses a two-stage high-confidence/low-confidence procedure in the style of ByteTrack and BoT-SORT. Geometry is scored with Distance IoU: $Y$1 where $Y$2 is the Euclidean distance between box centers and $Y$3 is the diagonal of the smallest box enclosing both candidate boxes. Appearance similarity from the learned embeddings is combined with DIoU when the detection confidence exceeds a threshold. This design is intended to reduce ID switches in occlusions and complex maneuvers while preserving partially visible targets through low-confidence association (Cheng et al., 2023).

Track management follows the standard tracking-by-detection cycle: unmatched high-confidence detections initialize tracks, matched tracks update the Kalman state and stored appearance representation, and tracks are terminated after a specified period of unmatched frames (Cheng et al., 2023).

6. Training protocol, benchmarks, and empirical results

CenterRadarNet is trained from scratch on K-Radar, a multimodal dataset containing 4D radar, LiDAR, and stereo camera data under clear, rainy, and snowy conditions, with varying illumination and road structures. The 3D detection split contains approximately 17.5k training frames and 17.5k test frames. For tracking, the work uses enhanced annotations on K-Radar V2 with re-annotated boxes and completed tracking IDs, excluding sequences 51, 52, 57, and 58 because LiDAR and camera are fully occluded there. The reported focus classes are “Sedan” and “Bus or Truck” (Cheng et al., 2023).

Training uses an NVIDIA V100 GPU, batch size 16, and 30 epochs. The maximum number of objects per frame in the detection head is 30, the embedding dimension is 32, and the JDE temporal window is $Y$4. The optimizer and learning-rate schedule are not explicitly specified, and no cross-modal pretraining is used (Cheng et al., 2023).

On the K-Radar 3D detection benchmark, CenterRadarNet reports Sedan $Y$5 and $Y$6, and Bus/Truck $Y$7 and $Y$8, under the K-Radar protocol using $Y$9, $X$0, and IoU threshold 0.3. These values exceed the cited radar point-cloud baselines RTNH and CenterPoint with PointPillars or VoxelNet backbones (Cheng et al., 2023).

Ablation results attribute the gains to three elements: 3D volumetric processing in HR3D, multi-resolution information exchange, and JDE training. The ablation also states that adding JDE does not degrade detection and slightly improves it while enabling re-ID. Among appearance objectives, triplet hard loss with cosine-similarity distance yields the strongest combined detection and tracking results (Cheng et al., 2023).

For tracking on K-Radar V2, the best reported configuration—triplet loss with cosine distance and DIoU-based association—achieves MOTA $X$1, IDF1 $X$2, FP $X$3, FN $X$4, and ID switches $X$5. DIoU consistently improves over IoU-only association, and qualitative examples indicate stable identities under different weather, traffic densities, occlusions, and complex motion. The full detection-plus-tracking pipeline is reported to run at approximately 23 FPS on an NVIDIA RTX 3090 GPU (Cheng et al., 2023).

7. Related systems, conceptual boundaries, and limitations

CenterRadarNet belongs to a lineage of center-based radar perception models, but its scope is narrower and more specific than the broader label might imply. A common source of confusion is the relationship to CRN, “Camera Radar Net for Accurate, Robust, Efficient 3D Perception.” In the CRN paper, “CenterRadarNet” does not appear anywhere as an official name, acronym, or module; CRN is instead a camera–radar fusion framework whose detection head is center-based and anchor-free, following CenterPoint (Kim et al., 2023). By contrast, CenterRadarNet in the strict sense denotes a radar-only framework operating on raw 4D FMCW tensors (Cheng et al., 2023).

Two adjacent precedents clarify this distinction. CenterFusion is a center-based radar–camera detector that first localizes image-plane centers, then performs frustum-based radar association and middle fusion to refine depth, rotation, and velocity (Nabati et al., 2020). RODNet is a radar-only detector that predicts range–azimuth confidence maps from RAMaps with a temporal 3D encoder–decoder, but it does not regress 2D or 3D bounding boxes and represents each object as a center point in radar BEV coordinates (Wang et al., 2020). This suggests that CenterRadarNet occupies a distinct design point: raw 4D radar tensors, direct full 3D box regression, and online tracking through jointly learned appearance embeddings.

The paper identifies several limitations. False positives can arise from highly reflective roadside structures such as guardrails and poles. Recognition performance drops for vehicles with unusual relative motion, such as perpendicular or turning motion at intersections. Objects with zero CFAR points are removed from training, which biases learning toward radar-visible targets. Elevation is present, but its resolution remains coarser than horizontal resolution, so fine-grained vertical localization is less precise. Suggested extensions include multi-sensor fusion, more explicit temporal modeling with temporal 3D CNNs, RNNs, or transformers, broader class sets beyond Sedan and Bus or Truck, and optimization of HR3D for embedded deployment through pruning or quantization (Cheng et al., 2023).