RAVEN: Radar Adaptive Vision Encoder Networks

• RAVEN is a deep-learning architecture that processes raw chirp-wise FMCW radar data to perform object detection and free-space segmentation with high efficiency.
• It employs independent state-space models per MIMO channel and a learnable cross-antenna mixer to disentangle spatial features while preserving phase coherence.
• An early-exit mechanism dynamically reduces latency by 82%, achieving state-of-the-art performance with 94.5% mAP and 89.5% mIoU on benchmark datasets.

RAVEN

RAVEN (Radar Adaptive Vision Encoder Networks) is a computationally efficient, end-to-end deep-learning architecture for fast-chirp-wise object detection and free-space segmentation using FMCW (Frequency-Modulated Continuous Wave) radar. It departs from traditional frame-based radar pipelines by processing raw ADC (analog-to-digital converter) data in a chirp-wise streaming fashion, preserving the full MIMO (Multiple-Input Multiple-Output) aperture through independent state-space encoders, and employing a learnable cross-antenna mixing module. RAVEN further incorporates an early-exit mechanism, enabling the model to reduce latency by making detection/segmentation decisions as soon as the latent state converges, often long before a full radar frame is sampled. The design achieves state-of-the-art detection and segmentation performance—94.5% mAP and 89.5% mIoU—while reducing computation by 82% compared to full-frame radar baselines (Sen et al., 6 Apr 2026).

1. Chirp-wise Streaming and MIMO SSM Encoders

RAVEN ingests each fast-time sample vector $u_t \in \mathbb{R}^L$ (with $L$ samples per chirp) as soon as it is acquired, foregoing the conventional approach of waiting for $N_c$ entire chirps to form a range–Doppler data cube. For a radar with $N_{tx}$ transmitters and $N_{rx}$ receivers, the total virtual array channels $N_v = N_{tx} \cdot N_{rx}$ are preserved by instantiating an independent state-space model (SSM) per Rx channel.

For each receive channel $i$, the SSM update is given by: $$x_i[t] = A_i x_i[t-1] + B_i u_i[t], \quad y_i[t] = C_i x_i[t] + D_i u_i[t]$$ where $x_i[t] \in \mathbb{R}^d$ is the hidden state (with $d$ typically 16), $L$0, $L$1, $L$2, and $L$3. The parameters are learned using the Mamba formulation.

The outputs $L$4 (K=1 "token" per channel per chirp) summarize instantaneous phase, range, and signal envelope, passing these features without loss of phase coherence to downstream processing (Sen et al., 6 Apr 2026).

2. Learnable Cross-Antenna Mixer and Spatial Tokenization

RAVEN fuses the per-channel SSM outputs $L$5 using a multiple-head attention mechanism: $L$6

$L$7

where $L$8 are learned. The softmax is applied row-wise to yield an $L$9 attention matrix. $N_c$0 is reshaped into a spatial feature map ($N_c$1) by a 1×1 convolution, before projection into the backbone. This cross-antenna mixer learns to disentangle azimuth, elevation, and Doppler signatures directly from the virtual array, promoting data-adaptive spatial encoding while maintaining permutation equivariance (i.e., no assumption of explicit array geometry) (Sen et al., 6 Apr 2026).

3. Early-Exit Mechanism for Latency Reduction

After each processed chirp, RAVEN computes a global feature $N_c$2 via pooling over $N_c$3. The cosine similarity to the previous global state,

$N_c$4

is evaluated. When $N_c$5 (with $N_c$6), the model early-exits, bypassing further chirps and immediately dispatching $N_c$7 to detection and segmentation heads. The threshold $N_c$8 is selected by minimizing a weighted combination of expected latency and task loss: $N_c$9 where $N_{tx}$0 is the number of chirps processed before exit. RAVEN thus dynamically short-circuits inference depending on the convergence of the latent feature trajectory, yielding substantial reductions in both computational volume and wall-clock latency while preserving or only moderately degrading accuracy (Sen et al., 6 Apr 2026).

4. End-to-End Pipeline and Inference Pseudocode

The complete pipeline, abstracted in pseudocode, is as follows: $N_{tx}$1 The early exit is determined solely by feature similarity, ensuring RAVEN can operate in a streaming, low-latency context with variable input length per inference (Sen et al., 6 Apr 2026).

5. Empirical Performance and Benchmark Results

RAVEN's performance is characterized by competitive detection and segmentation metrics, alongside marked computational efficiency:

• On the RaDICaL dataset (BEV detection and segmentation):
  • Joint detection+segmentation: mAP = 95.0%, mAR = 95.1%, F1 = 94.8%
  • BEV segmentation: mIoU = 90.2% (joint) vs 90.1% (single-task)
  • Classwise AP: Pedestrian 96.3%, Vehicle 98.4%
• Computational savings:
  • Mean chirps per frame: 8.2 (cosine early exit) vs. 128 in conventional frame-based, an 82% reduction in GMACs
  • Cosine-exit: 94.5% mAP, 89.5% mIoU; entropy-exit (alternative early-exit method): 93.6% mAP, 88.8% mIoU
• On public RADDet data (3Tx×4Rx, 128 chirps): F1 = 67.0% (vs 60.8% T-FFTRadNet)

These results are supported by comprehensive ablations—token count, mixer design, and early-exit thresholding—as well as detailed trade-off curves in Figures R1 and R4 of the cited paper (Sen et al., 6 Apr 2026).

6. Limitations and Directions for Further Research

• RAVEN's chirp-wise streaming assumes that targets are quasi-stationary across the processing window; highly dynamic (rapidly accelerating) objects can exhibit phase discontinuities and reduced Doppler resolution, particularly if early exit shortens integration time.
• The cross-antenna mixer is equivariant to channel order and does not encode array geometry directly; future extensions could inject geometric priors or positional embeddings inspired by subspace methods (e.g., ESPRIT).
• The current architecture is limited to 2D BEV tasks, as defined by the available benchmarks; extending RAVEN to 3D detection and segmentation, possibly with point-cloud or LiDAR supervision, remains an open research avenue.
• While adaptive chirp scheduling based on scene motion was explored, no straightforward link to object velocity was found; reinforcement learning or policy-driven chirp allocation may yield superior scheduling (Sen et al., 6 Apr 2026).

7. Summary and Significance

RAVEN presents a unified, SSM-driven, multi-task radar backbone for chirp-wise object detection and segmentation with a permutation-equivariant cross-antenna fusion module and dynamic early-exit controller. It achieves state-of-the-art accuracy and an over 80% reduction in computational cost and latency relative to full-frame radar pipelines. Its streaming, latency-minimizing design and preservation of phase- and array-coherent structure position it as a significant advance for low-latency radar perception, particularly in automotive and robotics contexts (Sen et al., 6 Apr 2026).