---
title: 4D mmWave Radar Dataset Overview
url: https://www.emergentmind.com/topics/4d-mmwave-radar-dataset
type: topic
---

# 4D mmWave Radar Dataset Overview

A 4D mmWave radar dataset comprises measurement data from automotive or robotic frequency-modulated continuous-wave (FMCW) radar sensors providing per-point range, azimuth, elevation, and Doppler velocity, captured in traffic, indoor, marine, or other operational scenarios. These resources serve as the foundational benchmarks for 3D object detection, multi-object tracking, SLAM, panoptic perception, and sensor fusion in conditions where cameras and LiDARs may be impaired by occlusion, weather, or limited illumination. Public 4D mmWave radar datasets typically package synchronized radar, camera, and optionally LiDAR data, annotated with object-level ground truth (e.g., 3D bounding boxes, tracks, or occupancy voxels), and span diverse environments including urban streets, highways, industrial zones, indoor spaces, and maritime domains.

## 1. Principles of 4D mmWave Radar Data Acquisition

4D mmWave radar systems sample the reflected RF spectrum via an FMCW chirp protocol and MIMO antenna array, enabling high-resolution estimation of:

- **Range ($r$)**: Determined via fast-time (range) FFT across ADC samples.
- **Azimuth ($\theta$) and Elevation ($\phi$)**: Recovered by digital beamforming (angle FFT or Capon/MUSIC) across virtual channels.
- **Doppler velocity ($v_D$)**: Extracted from slow-time (chirp) FFT, yielding relative radial velocities per point.

Typical raw outputs are either tensor-format (ADC cubes: $R[n_{r}, n_{d}, n_{a}, n_{e}]$), or post-processed, sparse point clouds with Cartesian $[x,y,z]$, $v_D$, and reflectivity attributes. Processing includes range/Doppler/angle FFTs, multichannel calibration, CFAR detection for clutter suppression, and clustering for point grouping. The radar hardware is characterized by parameters including carrier frequency (76–81 GHz), bandwidth (1–4 GHz), virtual channel count (16–128+), range and angular resolutions ($\sim$3–10 cm, 1–5°), and frame rates (5–20 Hz) [2509.18613] [2405.12821] [2503.24091].

## 2. Public Datasets: Scope, Modalities, and Specifications

### Object Detection and Tracking

Key public 4D mmWave radar datasets include:

| Dataset            | Sensor Model            | Az/El Res.   | Env.          | Labels                          | Frames      |
|--------------------|------------------------|--------------|---------------|----------------------------------|-------------|
| View-of-Delft (VoD)| ZF FRGen21             | 1–1.5°       | Urban, campus | 3D boxes, tracks (Car, Ped, Cyc) | 8,682       |
| TJ4DRadSet         | Oculii Eagle           | 2°           | Urban, highway| 3D boxes, tracks (Car, Ped, Cyc, Truck) | 7,757|
| K-Radar            | TI 4D @ 77 GHz         | 1°           | Adverse WX    | 3D boxes, tracks                 | 35,000      |
| aiMotive           | (unspecified)          | –            | Mixed WX      | 3D boxes, tracks                 | 26,500      |
| V2X-Radar          | Oculii/Arbe Phoenix    | $\sim$1.5°   | Urban/co-op.  | 3D boxes (Car, Ped, Cyc, Bus, Truck) | 20,000 |

VoD provides five-scan accumulated point clouds, enabling denser frames for LiDAR-comparable performance, while TJ4DRadSet offers sparser single-sweep data containing a wide elevation FOV and challenging illumination. Additional datasets target long-range, multi-agent, large-scale, or adverse weather coverage (e.g., L-RadSet, Dual Radar, Bosch Street) [2509.18613] [2306.04242] [2503.24091] [2411.10962].

### SLAM and Mapping

SLAM-oriented datasets such as MSC-RAD4R (urban/suburban, 90k frames), NTU4DRadLM, and DIDLM provide synchronized radar, LiDAR, GPS/IMU, and sometimes IR/depth modalities. They focus on large-scale localization, trajectory estimation, and mapping in adverse weather, low light, and challenging surfaces [2404.09622] [2405.09780].

### Special Contexts

WaterScenes (Oculii Eagle, 54k frames), RadarRGBD (TI AWR2243), and mmFall (TI AWR1843, indoor gait/fall detection) extend 4D mmWave datasets to maritime navigation, indoor RGB-D fusion, and health monitoring, respectively [2307.07102] [2505.15860] [2003.02386].

## 3. Data Structure, Annotation, and Calibration

Typical 4D radar point cloud data is organized as variable-length $N \times k$ arrays per frame (e.g., $N$ points $\times$ 7–9 features: $[x,y,z,v_D,$ RCS/SNR, angle, time, \dots]$), sometimes as higher-dimensional spectra. Annotation formats include:

- **3D bounding boxes:** $[x_c, y_c, z_b, l, w, h, \theta]$
- **Track IDs:** for temporal association and MOT
- **Per-point/voxel semantic labels:** WaterScenes, RadarRGBD
- **Explicit cross-calibration:** Rigid-body transforms between radar, camera, LiDAR, and vehicle/world frames; timestamp alignment via hardware trigger, GNSS/IMU synchronization

Ground truth is obtained either by co-registered LiDAR with manual annotation or by camera-guided 3D projection and review. Dataset splits (train/val/test) vary—some datasets (VoD, TJ4DRadSet) provide official partitions, others require protocol agreement [2509.18613] [2405.12821] [2411.10962].

## 4. Preprocessing and Mathematical Formulations

Common pipelines include:

- **Ego-motion compensation:** For multi-scan accumulation, radar points are compensated using synchronized vehicle pose estimates to suppress ghosting (VoD: five-scan accumulation) [2509.18613].
- **Hard spatial filtering:** Applied to all datasets prior to voxelization or grid construction.
- **Instance-guided densification:** Radar points projected onto camera or segmentation masks, with Gaussian/uniform sampling and nearest-neighbor depth assignment to augment sparse returns, producing virtual points for feature learning [2509.18613].
- **Voxelization and attention encoding:** Voxel grids (resolution $\Delta x, \Delta y, \Delta z$) hold limited point counts per cell, with triple-attention mechanisms learning point-, channel-, and voxel-wise weights for robust representation (Eqs. 8–14 in [2509.18613]).
- **Kernel density estimation (KDE):** Used in SMURF to construct multi-dimensional density features per point, denoising and compensating for scatter [2307.10784].
- **Clustering and filtering:** DBSCAN employed for static/dynamic separation (e.g., EFEAR-4D Doppler-circle fitting for ego-velocity) [2405.09780].
- **Projection and coordinate transforms:** Rigid and affine mappings standardize radar, LiDAR, and image geometry for fusion and annotation [2411.10962] [2509.18613].

## 5. Dataset Utility and Benchmark Performance

Fusion and radar-centric algorithms evaluate datasets using tasks such as 3D detection, tracking, panoptic segmentation, SLAM, and referring expression comprehension. Performance is reported as 3D mAP/BEV mAP at various IoU thresholds per class and area (e.g., entire annotated area [EAA], driving corridor [DCA]):

- **MLF-4DRCNet (VoD):** EAA mAP 60.28%, DCA mAP 82.57%; DCA mAP exceeds LiDAR PointPillars baseline [2509.18613].
- **SMURF:** Multi-representation fusion delivers state-of-the-art BEV/3D mAP on radar-only inputs [2307.10784].
- **RadarNeXt:** Demonstrates radar-only 3D mAP of 50.48 (VoD five-scan) and high inference throughput (>67 FPS on RTX A4000) [2501.02314].
- **Talk2Radar:** Establishes a benchmark for 3D referring expression comprehension on 4D radar with language grounding [2405.12821].
- **WaterScenes:** Multi-task segmentation (PointNet++) achieves mIoU ≈ 60.7% for point-cloud semantic segmentation [2307.07102].
- **DIDLM, MSC-RAD4R (SLAM):** Baselines are set for 3D mapping error (ATE ≈ 0.15 m/100 m, RMSE ≈ 0.20 m depending on sequence/weather) [2404.09622].

Reported results demonstrate 4D mmWave radar resilience, with only modest drops in adverse scenarios where camera/LiDAR degrade by 30–50% [2503.24091].

## 6. Expansion, Simulation, and Limitations

Efforts to overcome hardware and scenario limitations include:

- **Dataset simulation:** Neural pipelines (DIS-Net, RSS-Net) synthesize realistic 4D radar from LiDAR/camera, generating point clouds and signal strengths statistically matched to real data (Dis-Net KL divergence $D_{KL}=0.154$ on VoD) [2503.08068].
- **Scaling and Pretraining:** GRT collects 1M samples (29 h) of raw I/Q cube data as a foundation for large radar pretraining; ablations show amplitude-only, AoA, CFAR, and Doppler-omitting representations incur substantial accuracy loss [2509.12482].
- **Coverage gaps:** Many datasets omit full publication of RF parameters (bandwidth, FMCW chirp, angular resolution), limiting absolute performance comparison and low-level algorithmic development [2306.04242].
- **Sparsity and clutter:** Standalone radar performance is constrained by point sparsity, measurement noise, and multi-path/ghosting; modern pipelines use multi-modal densification, KDE filtering, and attention-based encoding as mitigation [2509.18613] [2307.10784].
- **Resolution constraints:** Even state-of-the-art single-chip radars deliver 8–12 azimuthal bins and coarse elevation, stressing the need for robust downstream architectures or fusion [2509.12482].

## 7. Access, Licensing, and Future Directions

The majority of published 4D mmWave radar datasets are available under research-consent, CC-BY, or academic licenses, with data and tooling (calibration, preprocessing, loader scripts) released on GitHub or institutional pages [2405.12821] [2505.15860] [2411.10962]. Ongoing development trends include:

- **Broader environment coverage:** New datasets target extreme weather, long-range, multi-agent (cooperative V2X), marine, and low-light scenarios [2503.24091].
- **Standardization:** Increased call for full hardware parameter disclosure (Tx/Rx count, chirp parameters, bandwidth) to support reproducibility and raw tensor/ADC access [2306.04242].
- **Raw data utility:** Foundational models leveraging raw complex I/Q cubes outperform traditional processed representations, with scaling laws indicating $10^8$–$10^9$ frames may be needed to fully saturate such models [2509.12482].
- **Annotation diversity:** Moving from bounding boxes and tracks to per-point/voxel semantic labels and bespoke tasks (e.g., 3D REC, panoptic segmentation) [2307.07102] [2405.12821].

Overall, 4D mmWave radar datasets have become indispensable in pushing the limits of autonomous perception, robust mapping, and intelligent multisensor fusion—particularly in challenging, safety-critical, and previously inaccessible domains [2509.18613] [2306.04242] [2503.24091].

Source: https://www.emergentmind.com/topics/4d-mmwave-radar-dataset