---
title: 'CRUW Dataset: Radar-Only Detection Benchmark'
url: https://www.emergentmind.com/topics/cruw-dataset
type: topic
---

# CRUW Dataset: Radar-Only Detection Benchmark

The CRUW dataset ("Camera–Radar Unifying Workbench") is a large-scale, publicly available resource designed to advance radar-only object detection (ROD) in autonomous driving scenarios. It consists of highly synchronized and calibrated data from automotive-grade frequency-modulated continuous-wave (FMCW) radar and stereo RGB cameras, accompanied by systematic per-frame annotations for vehicle, pedestrian, and cyclist objects. CRUW is foundational in enabling the development and evaluation of deep learning models that operate solely on radio frequency (RF) sensor data, a key requirement for robust perception in adverse weather and challenging visibility.

## 1. Sensor Configuration and Data Modalities

CRUW employs dual 77 GHz FMCW automotive radar antennas operating at a high 30 Hz frame rate, yielding robust RF returns with 256 chirps captured per frame, of which four representative chirps are typically used [indices: 0, 64, 128, 192]. The radar produces complex-valued Range–Azimuth (RA) heatmaps of size 128×128, with separate real and imaginary channels, delivering 0.23 m range resolution and 15° azimuth (angular) resolution. Synchronized stereo FLIR RGB cameras capture images at 1440×1080 px, 93.6° horizontal field of view, and a 0.35 m baseline—enabling high-accuracy extrinsic calibration and cross-modal annotation transfer [2105.05207].

<table>
  <tr>
    <th>Modality</th>
    <th>Resolution/Frequency</th>
    <th>Role</th>
  </tr>
  <tr>
    <td>Radar (FMCW, dual 77 GHz)</td>
    <td>128×128 RA, 30 Hz</td>
    <td>Primary—detection and localization (RA maps)</td>
  </tr>
  <tr>
    <td>RGB Cameras (FLIR stereo)</td>
    <td>1440×1080, 30 Hz</td>
    <td>Annotation propagation, 3D scene geometry</td>
  </tr>
</table>

The radar sensors are co-mounted and strictly hardware-synchronized with cameras, allowing deterministic frame alignment and robust multi-modal fusion during annotation.

## 2. Data Collection Scenarios and Dataset Composition

CRUW consists of approximately 400,000 time-aligned frame pairs, encompassing ∼3.5 hours of diverse driving data across 464 sequences. The scenarios include parking lots, campus roads, city streets, highways, and intersections. CRUW emphasizes breadth over rare event coverage: public splits focus on clear-weather, daytime conditions without heavy precipitation or fog. Roughly 260,000 object instances (Car, Pedestrian, Cyclist) are annotated within the radar field of view (0–25 m range, ±90° azimuth), with about 92% in training sequences and 8% reserved for testing [2105.05207, 2003.01816].

<table>
  <tr>
    <th>Scenario</th>
    <th>#Sequences</th>
    <th>#Frames</th>
    <th>Vision-Hard %</th>
  </tr>
  <tr>
    <td>Parking Lot</td>
    <td>124</td>
    <td>106,000</td>
    <td>15%</td>
  </tr>
  <tr>
    <td>Campus Road</td>
    <td>112</td>
    <td>94,000</td>
    <td>11%</td>
  </tr>
  <tr>
    <td>City Street</td>
    <td>216</td>
    <td>175,000</td>
    <td>6%</td>
  </tr>
  <tr>
    <td>Highway</td>
    <td>12</td>
    <td>20,000</td>
    <td>0%</td>
  </tr>
</table>

The average number of annotated objects per frame in the original release is approximately 0.65; later radar-centric object detectors (e.g., RadarFormer) report up to ≈5 objects per frame, depending on the detection protocol [2304.08447, 2003.01816].

## 3. Annotation Pipeline and Ground Truth Generation

CRUW uses a novel “camera–radar coordinate alignment” pipeline to generate radar-space labels without manual intervention [2105.05207, 2003.01816]. The multi-stage process includes:

1. **Camera-based detection**: Mask R-CNN detects and classifies objects in RGB frames, estimating 2D boxes, instance masks, and class labels.
2. **CFAR-based radar peak detection**: The Constant False Alarm Rate (CFAR) algorithm finds peaks in each radar frame, forming candidates in (range, azimuth) space.
3. **Bilateral coordinate projection**: Closed-form 3D transformations map detected objects between camera pixels and radar BEV (r, θ), with full derivations for both directions. This ensures geometric consistency by leveraging global extrinsic calibration ($T_{cr}$) and ground-plane optimization.
4. **Fusion and clustering**: Gaussian probability maps in RA space are computed for each detected object, integrating camera and radar peak information. Spatial clustering via DBSCAN refines the radar object centroids.
5. **Per-class “ConfMaps”**: Each frame receives a separate 2D confidence Gaussian map (128×128) per class, where the center denotes the (range, azimuth) cell of the object and the standard deviation $\sigma$ encodes its spatial scale.

All “vision-hard” frames (strong occlusion, night, glare, blur) and a 10% subset of standard test frames undergo manual verification and correction. The final annotation protocol provides class, 2D position, and auxiliary kinematic attributes (e.g., speed), supporting both detection and tracking tasks [2105.05207].

## 4. Data Structure, Preprocessing, and Access

Each CRUW sequence contains synchronized directories for left/right camera images, radar tensors, and annotation text files. Radar data are provided as pre-processed .npy tensors (range×azimuth×channels), while annotations follow a KITTI-style plain text format, enumerating frame index, unique object ID, class, spatial coordinates, velocities, and other properties.

Preprocessing for deep learning models involves the following pipeline [2304.08447]:
- Raw radar ADC data are reshaped to $B × 2 × T × C × H × W$ (batch size, 2 channels, temporal window, chirps, spatial dimensions).
- Channel–chirp–time merging (denoted “M-Net”) reduces the input dimension from $(2, C)$ to $C_h$ channels.
- Temporal downsampling (via 3D convolution) collapses $T \to 1$, accelerating inference with negligible AP loss (<1%).
- Input normalization is minimal; the pipeline uses raw magnitude intensities, with no log-scaling or standardization. CFAR is strictly an annotation tool.
- Post-backbone, the temporal dimension is restored for output “ConfMap” generation.

The dataset and code are distributed under a CC BY-NC-SA 4.0 license at https://www.cruwdataset.org/, with loading, visualization, and evaluation tools available at https://github.com/CRUWdataset/CRUW-Bench [2105.05207].

## 5. Evaluation Protocols and Metrics

CRUW evaluation centers on localization and classification. The core detection metric is the Object Location Similarity (OLS), which generalizes IoU to (range, azimuth) space:

\[
\mathrm{OLS}(i,j) = \exp\Bigl(-\frac{d_{ij}^2}{2(s_j \kappa_{cls})^2}\Bigr)
\]

where $d_{ij}$ is Euclidean separation in meters, $s_j$ the ground-truth distance to the sensor, and $\kappa_{cls}$ a class-specific tolerance [2105.05207, 2003.01816]. Detections are matched to ground truth by OLS (>0.5 considered a match).

For each class and for each OLS threshold $t \in \{0.5, 0.55, ..., 0.9\}$:
- Precision and recall are tabulated.
- Average Precision (AP) and Average Recall (AR) are computed as means over thresholds.
- Class-averaged mAP and mAR summarize detection performance.

Additional metrics include Mean Absolute Localization Error (MAE) and Detection Quality F1 (DQF1), which incorporates MAE, precision, and recall.

A representative benchmarking table [2003.01816]:

| Method             | overall AP | overall AR |
|--------------------|------------|------------|
| Decision Tree      | 4.70 %     | 44.26 %    |
| CFAR+ResNet        | 40.49 %    | 60.56 %    |
| CFAR+VGG16         | 40.73 %    | 72.88 %    |
| RODNet             | 83.76 %    | 85.62 %    |

These protocols enforce consistency with leading 2D/3D vision benchmarks while accounting for radar-specific ambiguity and noise [2105.05207, 2304.08447].

## 6. Comparative Analysis and Challenges

CRUW is notable for its high frame rate, large sample count, and minimalist, pure RA-space radar signal representation. It contrasts with other public radar datasets:
- **Radiate**: Multi-modal (radar, camera, lidar) with adverse weather, but more complex data formats (RA, RD, RAD).
- **voD**: Lower radar frame rate (10 Hz) and coarser angular bins; includes Doppler.
- **nuScenes**: Aggregates radar as pseudo-LiDAR point clouds in coarsely binned BEV format; no real/imaginary channel distinction; object maps restricted to a single class [2304.08447].

CRUW’s technical challenges include:
- Severe low SNR and speckle in RA heatmaps, requiring robust CFAR peak detection and strong model regularization.
- The absence of elevation (vertical) cues—returns are collapsed to BEV, hampering separation of overlapping objects in height.
- Temporal consistency is of minor importance; aggressive downsampling of temporal cues has minimal detrimental effect on detection accuracy.
- CNN-only architectures like RODNet experience high inter-run variance and instability, while transformer-based backbones exhibit more stable convergence and generally higher AP [2304.08447].

## 7. Applications, Extensions, and Future Directions

CRUW serves as the main benchmarking platform for radar-only perception models and has catalyzed the development of advanced detection architectures (e.g., RODNet [2003.01816], RadarFormer [2304.08447]). Its influence extends to:
- Enabling robust detection in weather-degraded visibility regimes, where optical sensors fail.
- Providing a reproducible platform for multi-object tracking, sensor fusion, and self-supervised learning research.
- Extension toward richer multi-modal datasets (e.g., CRUW3D [2311.10261]), which include LiDAR, expanded object classes (van, truck, bus), and broader sensor coverage.

Future improvements may entail higher-resolution Doppler data, more diverse environmental conditions, annotation of rare/edge-case events, and unified detection/tracking benchmarks. The pipeline and its systematic annotation methodology are directly transferable to new sites or sensor configurations, provided accurate synchronizations and calibrations are maintained.

CRUW remains a central asset underpinning progress in RF-centric autonomous navigation, radar deep learning, and robust, real-world urban perception.

Source: https://www.emergentmind.com/topics/cruw-dataset