RAD-GAN: Radar-Aware Speech Reconstruction

• The paper introduces a dual-conditioned two-stage pipeline that transforms noisy, band-limited radar signals into intelligible full-band (8 kHz) speech.
• It utilizes a Residual Fusion Gate, HiFi-GAN-based generator, and multi-mel discriminators to effectively fuse radar and WVN-derived mel spectrograms.
• Empirical results show that RAD-GAN outperforms several baselines, achieving superior weighted scores despite low SNR and challenging sensing conditions.

Searching arXiv for the RAD-GAN paper and closely related radar GAN work. arXiv search query: "Radar-Aware Dual-conditioned GAN mmWave radar speech reconstruction" Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN) denotes a two-stage speech reconstruction pipeline for mmWave radar in which a radar-derived low-band, low-SNR representation is transformed into an intelligible full-band $8$ kHz waveform through dual-conditioned adversarial training. In its named form, RAD-GAN was introduced for speech reconstruction from mmWave FMCW radar captures obtained through a glass wall, under global SNR roughly $-5$ dB to $-1$ dB, with a design centered on a Residual Fusion Gate (RFG), a Multi-Mel Discriminator (MMD), and staged training that first learns bandwidth extension from synthetically clipped clean speech and then adapts to real radar-derived inputs (Karani et al., 25 Feb 2026).

1. Definition and problem setting

RAD-GAN addresses a radar-conditioned speech reconstruction problem rather than conventional radar imaging or radar scene synthesis. The input is not an optical recording, nor a clean microphone spectrogram corrupted by additive noise, but a radar-derived observation whose information content is both band-limited and noisy, and whose relation to the target waveform is indirect because mmWave radar measures vibration rather than air pressure. The target is a full-band $8$ kHz waveform, while the reliable conditioning content is concentrated below about $1$ kHz; the method therefore treats the task as speech reconstruction via bandwidth extension (Karani et al., 25 Feb 2026).

The paper formulates two data-acquisition regimes. In Task 1, the radar captures speaker diaphragm vibrations directly through a glass wall. In Task 2, the radar captures vibrations from an aluminum foil placed near the speaker diaphragm, which yields worse segmental SNR. Across both tasks, the task difficulty is defined by three forms of mismatch: the radar input is low-SNR, spectrally limited, and collected under physically difficult sensing conditions. The paper therefore frames RAD-GAN not as a generic enhancement model, but as a constrained low-band-to-full-band reconstruction system that must preserve whatever low-frequency content is recoverable while inferring plausible upper-band structure (Karani et al., 25 Feb 2026).

A common misunderstanding is to treat RAD-GAN as a generic “radar GAN” for automotive perception, point-cloud generation, or radar-map translation. That interpretation is not supported by the named RAD-GAN paper. In this usage, RAD-GAN is specifically an mmWave radar-to-speech model. This distinguishes it from earlier radar GAN work on point-cloud scene synthesis (Nawaz et al., 2024), radar signal restoration (Zahid et al., 2024), one-bit PMCW range–Doppler reconstruction (Wang et al., 17 Mar 2025), or spectral recovery in UWB SAR (Tran et al., 2018).

2. Dual conditioning and generator architecture

The architecture is organized as a two-stage radar-to-speech reconstruction pipeline. A noisy radar-derived waveform is converted into a mel representation. In parallel, the same noisy waveform is processed by WaveVoiceNet (WVN) to produce an auxiliary enhanced waveform, which is also converted into a mel spectrogram. These two mel representations are then fused by the Residual Fusion Gate, and the fused mel spectrogram conditions a HiFi-GAN-based generator that synthesizes the reconstructed waveform (Karani et al., 25 Feb 2026).

The paper is explicit that the generator architecture itself is not modified from HiFi-GAN. It uses the original HiFi-GAN generator without architectural modification, with transposed-convolution upsampling and Multi-Receptive Field (MRF) residual fusion blocks. The generator input is an 80-bin mel spectrogram derived with $n_{\text{fft}} = 1024$, hop size $=128$, window size $=512$, Hann window, $n_{\text{mels}}=80$, $f_{\min}=0$ Hz, and $-5$0 Hz. There is no stochastic noise input; the waveform is generated directly from mel conditioning (Karani et al., 25 Feb 2026).

The term “dual-conditioned” has a specific meaning in this paper. It does not mean that the HiFi-GAN generator contains two separate conditioning branches injected into its internal blocks. Instead, dual conditioning occurs before the generator: the conditioning mel is constructed from two sources, the noisy radar mel $-5$1 and the WVN-enhanced mel $-5$2. The fused mel $-5$3 is then supplied as the single conditioning input to the unchanged generator. This resolves a second common misconception: RAD-GAN is dual-conditioned at the input-representation level, not via a custom dual-stream generator body (Karani et al., 25 Feb 2026).

The Residual Fusion Gate is defined as

$-5$4

where $-5$5 denotes channel-wise concatenation, $-5$6 is element-wise multiplication, $-5$7 is a learnable scalar logit, and $-5$8 is the sigmoid function (Karani et al., 25 Feb 2026).

This fusion rule gives the architecture its main radar-aware inductive bias. $-5$9 acts as the carry baseline, while $-1$0 is the residual correction proposed by WVN. The gate $-1$1 decides where that correction should be trusted. Because the input to the gate is $-1$2, and $-1$3, the gate performs a $-1$4 pointwise mapping. The paper states that this is applied frame-wise, performing cross-frequency mixing across mel bins without temporal smoothing. Both the gate bias and the scalar $-1$5 are initialized to $-1$6 so that early training remains conservative and does not over-trust the auxiliary WVN branch (Karani et al., 25 Feb 2026).

3. Discriminators, losses, and training procedure

RAD-GAN augments the standard HiFi-GAN waveform adversaries with a mel-domain discriminator. The model retains MPD and MSD from HiFi-GAN, then adds a Multi-Mel Discriminator (MMD) to judge realism directly in the time-frequency domain. In the accessible description, “multi-mel” refers not to multiple mel resolutions, but to two parallel mel discriminators with different normalization strategies: one uses spectral normalization and one uses weight normalization (Karani et al., 25 Feb 2026).

Each MMD branch processes mel inputs of shape $-1$7, with $-1$8, through the layer progression

$-1$9

using $8$0 convolutions, padding $8$1, strides $8$2, and LeakyReLU activations after each layer. The final output is a patch-level score map, and intermediate feature maps are retained for feature matching loss (Karani et al., 25 Feb 2026).

The generator is trained with radar-aware reconstruction losses in addition to adversarial terms. The paper defines a high-frequency-weighted mel loss: $8$3 with $8$4, where $8$5 for mel bins above a cutoff $8$6, and $8$7 otherwise. This weighting is intended to counter the tendency of the model to underfit upper-band reconstruction when the radar observation is informative mainly below $8$8 kHz (Karani et al., 25 Feb 2026).

A second reconstruction term is the multi-resolution STFT loss

$8$9

with $1$0. The MR-STFT implementation uses FFT sizes $1$1, hop sizes $1$2, window lengths $1$3, and spectral weights $1$4 (Karani et al., 25 Feb 2026).

The training procedure is explicitly staged. In Stage 1, the generator is pretrained alone on synthetically clipped clean speech using

$1$5

with no discriminators. In Stage 2, the pretrained generator is fine-tuned adversarially on real radar-derived fused mel spectrograms using

$1$6

where $1$7 contains MSD, MPD, and MMD. The text does not report the numerical value of $1$8 (Karani et al., 25 Feb 2026).

Optimization details are given separately for the three training components. The generator pretraining uses AdamW, $1$9, initial learning rate $n_{\text{fft}} = 1024$0, exponential scheduler $n_{\text{fft}} = 1024$1 each epoch, batch size $n_{\text{fft}} = 1024$2, and 66k steps. WVN is trained separately using Adam, learning rate $n_{\text{fft}} = 1024$3, batch size $n_{\text{fft}} = 1024$4, gradient accumulation $n_{\text{fft}} = 1024$5, and 30 epochs. RAD-GAN fine-tuning then uses AdamW, $n_{\text{fft}} = 1024$6, initial learning rate $n_{\text{fft}} = 1024$7, exponential scheduler $n_{\text{fft}} = 1024$8, batch size $n_{\text{fft}} = 1024$9, and 100k steps. The full model contains $=128$0 trainable parameters (Karani et al., 25 Feb 2026).

4. Data regime and empirical performance

The experiments use the RASE 2026 Challenge paired radar–speech dataset collected with a TI AWR2243BOOST mmWave FMCW radar. The radar captures are obtained through a glass wall. Task 1 contains $=128$1 paired samples, with $=128$2 for training and $=128$3 for validation. Task 2 contains $=128$4 paired samples, with $=128$5 for training and $=128$6 for validation. The sampling rate is 8 kHz, the average duration is 6.4 s, and all samples are clipped to 4 s segments for training. The total paired duration is about 42 h (Karani et al., 25 Feb 2026).

The paper compares RAD-GAN against six baselines: WaveVoiceNet (M0), HiFi-GAN (M1), DCCTN (M2), AP-BWE (M3), DiffWave (M4), and CDiffuSE (M5). Evaluation uses PESQ, ESTOI, CSMFCC, and DNSMOS, with normalized task scores defined by

$=128$7

$=128$8

Task 2 is weighted more heavily because it is the harder setting (Karani et al., 25 Feb 2026).

RAD-GAN attains the best overall score among the reported systems. Its metrics are PESQ $=128$9, ESTOI $=512$0, CSMFCC $=512$1, DNSMOS $=512$2, Task 1 score $=512$3, Task 2 score $=512$4, and Weighted score $=512$5. The directly compared baselines obtain weighted scores of 0.260 for WaveVoiceNet, 0.288 for HiFi-GAN, 0.172 for DCCTN, 0.165 for AP-BWE, 0.106 for DiffWave, and 0.119 for CDiffuSE (Karani et al., 25 Feb 2026).

The ablation study is organized as B0 through B3. B0 is original HiFi-GAN and yields Weighted score $=512$6. B1, adding MMD + MR-STFT, yields 0.290. B2, further adding pretraining, yields 0.312. B3, further adding WVN conditioning, yields 0.333. The paper interprets this as showing that pretraining gives the largest single jump, while the fusion-based auxiliary conditioning provides a further gain. It also notes that individual metrics do not improve monotonically, whereas the weighted score increases steadily across the ablation ladder (Karani et al., 25 Feb 2026).

Qualitatively, the paper reports that RAD-GAN reconstructs clearer upper-band harmonics, preserves a silence region around 2.6–3.2 s with less leakage, follows the clean waveform envelope more closely, and produces sharper onsets/offsets and stronger peaks than WVN in the harder Task 2 examples. This suggests that the model is not merely suppressing noise but performing structurally guided bandwidth extension from weak low-band cues (Karani et al., 25 Feb 2026).

5. Position within radar generative modeling

RAD-GAN belongs to a broader family of radar-domain adversarial models, but its combination of speech reconstruction, dual conditioning, and radar-aware mel-domain discrimination is distinct. Earlier work established several partial precedents. SARGAN learned recovery of missing spectral information in UWB SAR by combining a masked Fourier-domain content loss with an adversarial prior, but its sole test-time condition is the corrupted signal itself and it does not provide explicit dual conditioning (Tran et al., 2018). BRSR-OpGAN performs blind restoration of raw complex I/Q radar waveform segments with a dual domain loss in the temporal and spectral domains, yet this is better understood as single-input conditional restoration with dual-domain supervision rather than a dual-conditioned design (Zahid et al., 2024).

In automotive radar, “Generative Adversarial Synthesis of Radar Point Cloud Scenes” established an unconditional PointNet++-based GAN for full radar point cloud scene synthesis, with a global discriminator and six segment-wise discriminators encoding range-dependent density structure. That work is explicitly unconditioned, and its relevance to RAD-GAN lies in radar-aware adversarial priors rather than dual conditioning (Nawaz et al., 2024). Likewise, “High-Resolution Range-Doppler Imaging from One-Bit PMCW Radar via Generative Adversarial Networks” addressed reconstruction of high-quality range–Doppler maps from one-bit PMCW measurements, but the described methods are a hybrid radar-processing-plus-GAN denoiser and an end-to-end learned RD reconstruction GAN rather than a named RAD-GAN system (Wang et al., 17 Mar 2025).

A concise comparison is as follows.

Paper	Primary target	Conditioning character
RAD-GAN (Karani et al., 25 Feb 2026)	mmWave radar-to-speech reconstruction	Dual-conditioned through fused noisy mel and WVN mel
BRSR-OpGAN (Zahid et al., 2024)	Blind restoration of raw radar I/Q segments	Conditional on corrupted waveform; dual-domain loss
Radar point-cloud GAN (Nawaz et al., 2024)	Automotive radar scene synthesis	Unconditional
One-bit PMCW RD GAN (Wang et al., 17 Mar 2025)	Range–Doppler reconstruction	Conditional reconstruction from degraded radar measurements
SARGAN (Tran et al., 2018)	Missing-spectrum UWB SAR recovery	Conditional on corrupted radar data

This lineage shows that RAD-GAN should not be read as an isolated invention detached from prior radar GAN work. A plausible implication is that it inherits two well-established radar-GAN themes—adversarial priors for ill-posed recovery and conditioning on degraded radar observations—while shifting them into the radar-to-speech domain and making the conditioning explicitly dual through pre-generator fusion (Karani et al., 25 Feb 2026).

6. Limitations, interpretation, and future directions

Several limits are explicit. The system is built around a very specific low-band radar regime in which useful information is concentrated below $=512$7 kHz. The dataset is relatively small compared with standard speech corpora, the method is tied to a glass-wall mmWave capture setup, and the full model is large at about 87M parameters. The paper does not report latency, so real-time deployment remains open (Karani et al., 25 Feb 2026).

The authors identify two direct future directions: reporting and optimizing real-time latency, and model compression through distillation for edge inference. They also emphasize that the system was trained on a limited dataset, with no pre-trained modules for RAD-GAN itself and no data augmentations, while still outperforming the compared approaches for the specific task. That claim should be interpreted in the scope provided by the paper: the result concerns the reported RASE 2026 Challenge setting rather than a broad claim over all radar-conditioned speech reconstruction problems (Karani et al., 25 Feb 2026).

A final interpretive point concerns the meaning of “radar-aware.” In RAD-GAN, radar awareness does not come from an explicitly physics-embedded generator. The generator is the original HiFi-GAN. Radar awareness instead appears in the problem formulation and the conditioning and loss design: low-band mel conditioning with $=512$8 Hz, high-frequency-weighted reconstruction loss, conservative residual fusion of noisy and WVN-derived mel features, and mel-domain adversarial supervision tailored to spectral errors that dominate under radar capture. This suggests a broader definition of radar-aware adversarial modeling: not necessarily a bespoke radar backbone, but a generative system whose conditioning pathway, discriminators, and optimization are explicitly matched to the information bottlenecks of the radar observation (Karani et al., 25 Feb 2026).