Generalizable Radar Transformer (GRT)

• Generalizable Radar Transformer (GRT) is a foundational encoder–decoder Transformer that processes raw 4D FMCW radar data to generate dense spatial predictions rivaling higher-resolution sensors.
• It tokenizes 4D radar cubes primarily over range and Doppler while preserving raw amplitude and phase, avoiding lossy CFAR and AoA preprocessing.
• GRT supports multiple tasks—3D occupancy, BEV occupancy, semantic segmentation, and ego-motion—across varied environments including indoor handheld, outdoor, and bike-mounted settings.

to=arxiv_search.search 彩神争霸大发快_json_string={"^^^^^^^^2query2 OR ti:\2^^^^^^^^"Towards Foundational Models for Single-Chip Radar\"","max_results":5,"sort_by":"relevance"} 天天中彩票ာ္ to=arxiv_search.search 弘鼎_json_string={"^^^^^^^^2query2 OR ti:\2query2^^^^,"sort_by":"relevance"} RTLRAV无码 to=arxiv_search.search _植物百科通_json_string={"^^^^^^^^2query2 Gaussian Radar Transformer Semantic Segmentation in Noisy Radar Data","max_results":^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2^^^^,"sort_by":"relevance"} สูตรบาคาร่า to=arxiv_search.search 天天中彩票上_json_string={"^^^^^^^^2query2 UNRIO GRT architecture raw mmWave radar IQ","max_results":^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2^^^^,"sort_by":"relevance"} to=arxiv_search.search 大发快三豹子_json_string={"^^^^^^^^2query2 Radar Transformer mmWave radar foundational models","max_results":5,"sort_by":"relevance"} Generalizable Radar Transformer (GRT) is a foundational encoder–decoder Transformer trained directly on raw 4D FMCW single-chip mmWave radar data to produce dense spatial predictions that rival much higher-resolution sensors. It was introduced together with I/Q-^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^M, a ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^,^^^^2query2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^5,^^^^2query2query2query2^^^^-frame, approximately 29-hour synchronized radar–lidar–camera dataset, and supports 3D occupancy prediction, BEV occupancy, semantic segmentation, and ego-motion while generalizing across indoor handheld, outdoor handheld, and bike-mounted settings (&&&^^^^2query2^^^^&&&).

^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^. Scope, motivation, and problem setting

GRT addresses a persistent limitation of single-chip mmWave radar: poor angular resolution in azimuth and elevation despite robustness to occlusions, darkness, and adverse environmental conditions. The underlying sensor class is compact and inexpensive, and it directly measures range and radial velocity, but its effective aperture and limited virtual array produce coarse angular bins, often on the order of approximately PRESERVED_PLACEHOLDER_^^^^2query2^^^^. The 2^^^^2query22^^^^5 formulation of GRT targets this regime explicitly by learning from raw I/Q-derived radar cubes rather than from post-processed point clouds or heatmaps (&&&^^^^2query2^^^^&&&).

A central motivation is that common radar preprocessing pipelines based on CFAR and AoA are lossy. They discard amplitude/phase structure and much of the dense spectral information available before thresholding and peak picking. GRT therefore treats the raw radar tensor as the primary learning object and frames radar perception as a pretrainable foundation-model problem rather than as a collection of isolated task-specific models. The associated dataset, I/Q-^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^M, spans indoor, outdoor, and bike settings with geographically disjoint test sets and different radar modulations, which is intended to support pretraining and cross-domain transfer (&&&^^^^2query2^^^^&&&).

The training corpus is organized into three principal environments: indoor handheld with 3^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2^^^^k frames and 8.9 hours, outdoor handheld with 372k frames and ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2^^^^.7 hours, and bike-mounted with 333k frames and 9.3 hours. Each setting has distinct operating statistics, including different range and Doppler resolutions. For example, the indoor configuration has PRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^ cm and $\Delta v \approx 3.8$ cm/s, whereas the outdoor and bike configurations have $\Delta R \approx 8.7$ cm; the bike subset has $\Delta v \approx 24.9$ cm/s and $v_{\max} \approx 8.0$ m/s (&&&^^^^2query2^^^^&&&).

2. Radar signal representation and tokenization

The input modality is FMCW single-chip radar collected with a TI AWR^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^843Boost via DCA^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2query2query2^^^^EVM at 2^^^^2query2^^^^ Hz. Raw I/Q ADC samples are acquired over fast time and slow time across TX$\times$RX channels to form a virtual antenna array. A standard signal chain converts these measurements into a dense complex 4D radar cube indexed by range $r$, Doppler/velocity $v$, azimuth $\theta$, and elevation PRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2^^^^. In the reported configuration, the cube size is

PRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^

and it is provided to the model as two channels, amplitude and phase (&&&^^^^2query2^^^^&&&).

The preprocessing pipeline consists of optional windowing, range FFT per chirp, Doppler FFT across slow time, and angle FFT or beamforming across the virtual antenna array. This produces a representation that preserves raw amplitude/phase information without CFAR thresholding or AoA peak picking. The associated radar equations are stated explicitly: PRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\22^^^^ In the dataset, PRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^3 GHz and PRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^4 mm (&&&^^^^2query2^^^^&&&).

GRT tokenizes the PRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^5 cube into 2^^^^2query2^^^^48 patch tokens using a linear projection of patches of size PRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^6 over rangePRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^7DopplerPRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^8azimuthPRESERVED_PLACEHOLDER_^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^9elevation. The azimuth and elevation axes are “patched out”: they are absorbed داخل each token rather than unfolded into a long sequence. Positional encodings are normalized by axis length, which allows resolution changes without rearchitecting the model. A key empirical result is that this range–Doppler-centric tokenization is preferable to alternatives that retain all axes symmetrically in the patch layout (&&&^^^^2query2^^^^&&&).

The ablation evidence is numerically specific. Relative to the best patching strategy, “4D patches” incur $\Delta v \approx 3.8$^^^^2query2^^^^ loss, range–azimuth–elevation incurs $\Delta v \approx 3.8$^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^, and Doppler–azimuth–elevation incurs $\Delta v \approx 3.8$2. This supports the design choice of patching primarily over range and Doppler while treating the low-resolution angle axes as local within-token structure (&&&^^^^2query2^^^^&&&).

3. Encoder–decoder design and output parameterization

Architecturally, GRT is a direct adaptation of ViT with a decoder, described as Perceiver I/O-like ^^^^^^^^2query2^^^^^^^^ injection. It uses full self-attention rather than windowed or axial sparsity, together with pre-norm layers, GeLU activations, dropout $\Delta v \approx 3.8$3, expansion ratio $\Delta v \approx 3.8$4, and sinusoidal positional encodings in both encoder and decoder (&&&^^^^2query2^^^^&&&).

After patch embedding, a learned output token is concatenated to the encoder input. The encoder’s output at this token is then tiled to the target output grid and augmented by a 3D sinusoidal positional encoding. That tiled tensor serves as the decoder ^^^^^^^^2query2 which cross-attends to the encoder outputs. The resulting mechanism performs a learned change of basis from radar-coordinate inputs to the spatial coordinate system required by the downstream task (&&&^^^^2query2^^^^&&&).

The model supports multiple task heads. For 3D occupancy, the output is a polar range–azimuth–elevation grid of size $\Delta v \approx 3.8$5 after decimation, with ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query22^^^^4 cubic patches of size $\Delta v \approx 3.8$6. The loss is a weighted binary cross-entropy corrected for polar cell size: $\Delta v \approx 3.8$7 For BEV occupancy, the output is a $\Delta v \approx 3.8$8 range–azimuth grid with ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query22^^^^4 patches of size $\Delta v \approx 3.8$9, optimized with the RadarHD-style mixture

$\Delta R \approx 8.7$^^^^2query2^^^^

For semantic segmentation, the output is a $\Delta R \approx 8.7$^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^ azimuth–elevation grid with 8 coarse classes and ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query22^^^^4 patches of size $\Delta R \approx 8.7$2, trained with per-class BCE on one-hot targets. For ego-motion, the head is a 3-layer MLP with 5^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\22^^^^-unit hidden layers and loss

$\Delta R \approx 8.7$3

with $\Delta R \approx 8.7$4 Doppler bins for numerical stability (&&&^^^^2query2^^^^&&&).

A family of model scales is reported.

Variant	Parameters	Throughput on RTX 4^^^^2query2sort_by2query2^^^^
pico	3.9M	75^^^^2query2^^^^ fps
tiny	^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\22^^^^.7M	32^^^^2query2^^^^ fps
small	28.9M	^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2(Huang et al., 15 Sep 2025)2query2^^^^ fps
medium	69.4M	84 fps
large	^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^49M	44 fps

These variants span $\Delta R \approx 8.7$5 to $\Delta R \approx 8.7$6 encoder/decoder layers and hidden dimensions from 256 to 768. The reported throughputs are batched throughput values (&&&^^^^2query2^^^^&&&).

4. Objectives, pretraining regime, and optimization

Training uses AdamW with batch size 32, learning rate $\Delta R \approx 8.7$7, and ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2query2^^^^-step warmup. Optimization terminates when validation loss plateaus, defined as three consecutive non-improving checkpoints with two checkpoints per epoch. The baseline pretraining strategy is self-supervised 3D occupancy prediction using lidar-derived labels, followed by fine-tuning for BEV occupancy, semantic segmentation, or ego-motion. A multi-task objective is possible,

$\Delta R \approx 8.7$8

but the reported experiments typically train tasks separately after 3D occupancy pretraining (&&&^^^^2query2^^^^&&&).

The labels are modality-specific. Occupancy labels are obtained by projecting lidar into polar grids, with explicit correction for polar cell area proportional to $\Delta R \approx 8.7$9. Semantic segmentation labels are camera-derived pseudo-labels produced by SegFormer-B5 trained on ADE2^^^^2query2^^^^k and aggregated to 8 coarse classes: person, sky, vehicle, flat, nature, structure, ceiling, and object. Ego-motion supervision is obtained from Cartographer SLAM fusing lidar and IMU, with approximately $\Delta v \approx 24.9$^^^^2query2^^^^ invalid segments pruned due to SLAM failures (&&&^^^^2query2^^^^&&&).

The model’s scaling behavior is summarized by a logarithmic trend: across ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2^^^^%, 2^^^^2query2^^^^%, 5^^^^2query2^^^^%, and ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2query2^^^^% training subsets, test losses improve by approximately $\Delta v \approx 24.9$^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^ per $\Delta v \approx 24.9$2 increase in data. The paper gives a convenient parametric form,

$\Delta v \approx 24.9$3

Larger models show slightly better data efficiency, but model-size scaling is described as weak at the current data scale. A rough estimate is that approximately ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2query2^^^^M samples, or around 3^^^^2query2query2query2^^^^ hours, would be required to fully exploit the model family’s potential (&&&^^^^2query2^^^^&&&).

The ablations emphasize preservation of raw spectral structure. Replacing the dense antenna representation with AoA causes $\Delta v \approx 24.9$4 loss, and CFAR thresholding with $\Delta v \approx 24.9$5 causes $\Delta v \approx 24.9$6 loss. Removing the Doppler FFT gives $\Delta v \approx 24.9$7, and shuffling slow time gives $\Delta v \approx 24.9$8. These degradations are interpreted in the paper as equivalent to more than a $\Delta v \approx 24.9$9 reduction in training data under the observed scaling law (&&&^^^^2query2^^^^&&&).

5. Empirical performance, transfer, and cross-setting generalization

For GRT-small after fine-tuning, the reported mean performance with $v_{\max} \approx 8.0$^^^^2query2^^^^ confidence intervals is as follows: 3D occupancy chamfer of 4.7 range bins $v_{\max} \approx 8.0$^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^, with per-setting values of ^^^^2query2^^^^.24 m $v_{\max} \approx 8.0$2 indoor, ^^^^2query2^^^^.4^^^^2query2^^^^ m $v_{\max} \approx 8.0$3 outdoor, and ^^^^2query2^^^^.38 m $v_{\max} \approx 8.0$4 bike; semantic segmentation mIoU of $v_{\max} \approx 8.0$5, accuracy $v_{\max} \approx 8.0$6, and top-2 accuracy $v_{\max} \approx 8.0$7; BEV chamfer of ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^ bins $v_{\max} \approx 8.0$8, with ^^^^2query2^^^^.28 m $v_{\max} \approx 8.0$9 indoor, ^^^^2query2^^^^.84 m $\times$^^^^2query2^^^^ outdoor, and ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^.3 m $\times$^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^ bike; and ego-motion error of ^^^^2query2^^^^.95 Doppler bins $\times$2 in speed and $\times$3 in angle (&&&^^^^2query2^^^^&&&).

Joint training across indoor, outdoor, and bike data improves over setting-specific training. Relative to joint training, indoor-only incurs $\times$4, outdoor-only $\times$5, and bike-only $\times$6 loss. Fine-tuning from the 3D occupancy base is reported to improve data efficiency by up to a $\times$7 effective increase in dataset size, with the strongest effect for semantic segmentation. The benefit tapers for BEV as data increases but persists for segmentation across scales (&&&^^^^2query2^^^^&&&).

On ColoRadar, approximately 3^^^^2query2^^^^ minutes of fine-tuning allow GRT to adapt to a $\times$8 resolution not present in pretraining and achieve geometric mean chamfer ^^^^2query2^^^^.98 m, surpassing an efficient cross-modal diffusion model at ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^.^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^9 m and RadarHD at ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^.73 m. The model also outperforms task-specific baselines for 2D BEV occupancy, specifically U-Net RadarHD and Swin-based T-FFTRadNet, across all training sizes (&&&^^^^2query2^^^^&&&).

Several qualitative claims are also reported. GRT produces sharper and less noisy depth maps, cleaner segmentation, and more accurate BEV occupancy, including cases where pedestrians are separated via micro-Doppler and ground is distinguished from vegetation through specularity. These statements are qualitative rather than metric-based, but they align with the numerical ablations showing the importance of Doppler and raw spectral inputs (&&&^^^^2query2^^^^&&&).

6. Limitations, deployment considerations, and open directions

Despite its generalization results, GRT remains constrained by the underlying single-chip aperture. Elevation and azimuth resolution are capped by the hardware, and the paper states that even with Doppler, elevation super-resolution is limited, especially near the vertical field-of-view edges. Supervision is also imperfect: semantic labels come from camera-derived pseudo-labels, occupancy labels inherit lidar range limits of roughly 2^^^^2query2^^^^–25 m, and I/Q-^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^M covers daylight and fair-weather conditions with one radar hardware family (&&&^^^^2query2^^^^&&&).

The authors also note that radar artifacts such as bleed, sidelobes, multipath, occlusions, and clutter can induce hallucinations. These observations delimit the empirical scope of the reported results. A plausible implication is that some of the remaining performance ceiling is label- and domain-limited rather than purely model-limited, but the paper does not quantify that decomposition (&&&^^^^2query2^^^^&&&).

From a deployment perspective, the reported compute budget is moderate by modern accelerator standards. GRT-small trains in approximately 22 RTX 4^^^^2query2sort_by2query2^^^^-hours, with total experimental compute around 874 RTX 4^^^^2query2sort_by2query2^^^^-hours. The reported inference throughput of about ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2(Huang et al., 15 Sep 2025)2query2^^^^ fps for the 28.9M-parameter small model indicates that raw-cube inference is not restricted to offline processing. The practitioner guidance is explicit: prefer raw 4D complex cubes over CFAR/AoA representations, preserve Doppler, patch primarily along range and Doppler, and use self-supervised 3D occupancy pretraining before downstream fine-tuning (&&&^^^^2query2^^^^&&&).

The future directions named in the paper include larger-scale pretraining at or above ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2query2query2^^^^M samples, multi-radar and multi-antenna configurations, explicit Doppler-aware modules, masked modeling for radar tensors, cross-modal contrastive or ranking objectives, and physics-informed augmentation and calibration. These extensions are framed as ways to push beyond the current data and sensor limits rather than as established properties of the present model (&&&^^^^2query2^^^^&&&).

7. Related architectures, nomenclature, and downstream adaptations

The acronym GRT is not unique in radar learning. In 2^^^^2query222^^^^, “Gaussian Radar Transformer” introduced a distinct point-based architecture for semantic segmentation in sparse automotive radar point clouds, operating on single scans from RadarScenes rather than on raw FMCW cubes. That model uses a Gaussian transformer layer in which softmax normalization is replaced by a channel-wise Gaussian kernel $\times$9, together with attentive downsampling and upsampling modules. On RadarScenes test data, it reports $r$^^^^2query2^^^^ and $r$^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^ for six classes under single-scan inference (Zeller et al., 2022).

The two models therefore differ in both representation and objective. Gaussian Radar Transformer is a radar-point-cloud segmentation network built around local vector attention and sparse geometry preservation; Generalizable Radar Transformer is a ViT-style foundational encoder–decoder trained on dense raw 4D spectral cubes for occupancy, segmentation, and motion-related tasks. The shared acronym can create ambiguity in citations, but the papers are technically separate and address different stages of the radar processing stack (Zeller et al., 2022).

A later extension appears in UNRIO, which builds a transformer-based radar-inertial odometry system on the GRT architecture for raw mmWave radar IQ. UNRIO converts the complex spectrum to three channels $r$2, collapses elevation and azimuth into the channel dimension, patches over Doppler and range, and uses a 4-layer Transformer encoder with decoder heads for depth, Doppler, direct body-frame velocity, and uncertainty. Training proceeds in three stages: geometric pretraining on LiDAR-projected depth, velocity or Doppler fine-tuning, and uncertainty calibration via heteroscedastic Gaussian NLL (Huang et al., 15 Apr 2026).

UNRIO’s downstream estimator maps predicted per-angle-bin Doppler to body-frame velocity via weighted least squares,

$r$3

with weights derived from predicted log-variances. Those uncertainty estimates are then propagated into a 3-second fixed-lag pose graph that fuses radar velocity factors with IMU preintegration. On IQ^^^^2id:(Huang et al., 15 Sep 2025) OR ti:\2^^^^M indoor test environments, the paper reports the lowest relative pose error on the majority of sequences, with especially strong gains on lateral-motion trajectories where classical DSP point-cloud baselines degrade (Huang et al., 15 Apr 2026).

Taken together, these works place Generalizable Radar Transformer within a broader shift in radar learning from hand-engineered thresholded representations toward end-to-end models operating on dense spectral structure. This suggests that the core GRT design—raw-spectrum tokenization, Transformer context aggregation, and flexible decoder queries—functions not only as a scene-understanding model but also as a reusable radar representation backbone across perception and odometry tasks (&&&^^^^2query2^^^^&&&).