Radar Diffusion Transformer for Respiration Sensing

• RDT is a specialized learnable denoiser in the mmWave-Diffusion framework that refines radar phase observations using an observation-consistent initialization.
• It integrates Transformer self-attention with banded-mask multi-head cross-attention and dual positional encodings to enforce strict temporal alignment.
• Ablation studies show that coupling local physical priors with conditional tokens significantly boosts waveform reconstruction and respiratory-rate estimation.

Searching arXiv for the specified paper and any directly relevant supporting diffusion reference. Radar Diffusion Transformer (RDT) is the learnable denoiser $\epsilon_{\theta}(\,\cdot\,)$ in the reverse diffusion process of mmWave-Diffusion, a framework for respiration sensing from millimeter-wave radar phase observations (Wang et al., 21 Mar 2026). Within that framework, RDT is explicitly conditioned on radar phase observations, initialized in an Observation-Consistent Neighborhood (OCN) around the radar phase rather than from pure Gaussian noise, and structured to preserve strict temporal correspondence between noisy respiration estimates and phase measurements (Wang et al., 21 Mar 2026). The architecture combines Transformer-style self-attention, banded-mask multi-head cross-attention, timestep-modulated normalization, and patch-level dual positional encodings to support micromotion interference removal in 20 reverse steps, with reported gains in waveform reconstruction and respiratory-rate estimation on synchronized radar-respiration data (Wang et al., 21 Mar 2026).

1. Functional role within mmWave-Diffusion

RDT serves as the learnable denoiser $\epsilon_{\theta}(\,\cdot\,)$ in the reverse diffusion process. At each reverse step, it takes as input the current noisy estimate of the respiration waveform $x_t$, the radar phase observation $y$, and the timestep index $t$, and outputs a prediction of the “clean” respiration residual used to form the Gaussian mean of $x_{t-1}$ (Wang et al., 21 Mar 2026). In the formulation described for mmWave-Diffusion, $x_t$ is initialized in an Observation-Consistent Neighborhood around $y$, rather than being generated from unstructured Gaussian noise.

The observation-anchored initialization is given as

$$x_{T} \;=\; y \;+\; k\,\sqrt{n_{T}}\;\epsilon,\quad \epsilon\sim\mathcal{N}(0,I),$$

with $n_T \approx 1$ (Wang et al., 21 Mar 2026). This differs from standard diffusion formulations that start from pure Gaussian noise. The stated purpose is to align the reverse trajectory with the actual radar measurement and bootstrap sampling in a physically consistent neighborhood. At each of the 20 learned reverse steps, RDT remains explicitly conditioned on $\epsilon_{\theta}(\,\cdot\,)$0, which is described as keeping the generative path anchored to the real observation and accelerating micromotion-artifact removal (Wang et al., 21 Mar 2026).

The broader conditional diffusion model is also defined through a forward perturbation kernel:

$\epsilon_{\theta}(\,\cdot\,)$1

A plausible implication is that RDT is not merely estimating a generic denoising direction, but operating within a diffusion process whose stochastic dynamics are already biased toward the observation $\epsilon_{\theta}(\,\cdot\,)$2, thereby reducing the mismatch between the model’s latent trajectory and the underlying sensing process.

2. Architectural composition

RDT is a small Transformer-style network composed of $\epsilon_{\theta}(\,\cdot\,)$3 identical blocks, with $\epsilon_{\theta}(\,\cdot\,)$4 in the reported experiments (Wang et al., 21 Mar 2026). Each block contains four core components: a timestep-modulated LayerNorm, multi-head self-attention on the main token stream, banded-mask multi-head cross-attention from the conditional phase stream, and a feed-forward network (FFN) with residual connections (Wang et al., 21 Mar 2026).

The timestep embedding is implemented as an MLP that maps the scalar $\epsilon_{\theta}(\,\cdot\,)$5 to a vector of size $\epsilon_{\theta}(\,\cdot\,)$6, and this vector is used to modulate the scales and biases in LayerNorm (Wang et al., 21 Mar 2026). The FFN is a two-layer MLP with GeLU activation and is applied after cross-attention in each block. Residual connections follow a standard add-and-norm structure around self-attention, cross-attention, and the FFN (Wang et al., 21 Mar 2026).

The functional division between self-attention and cross-attention is central. Self-attention operates on the main token stream associated with the current noisy respiration estimate $\epsilon_{\theta}(\,\cdot\,)$7, while cross-attention brings in information from the conditional token stream derived from the radar phase observation $\epsilon_{\theta}(\,\cdot\,)$8 (Wang et al., 21 Mar 2026). This separation allows RDT to model dependencies internal to the evolving waveform representation while simultaneously grounding that representation in measurement-conditioned features.

This design suggests that RDT occupies an intermediate position between a generic sequence denoiser and a physics-informed conditional generator. It retains the flexible token-mixing behavior of Transformer blocks, but uses the phase observation as a continuously available conditioning signal rather than as a one-time side input.

3. Patch-level dual positional encodings and temporal alignment

Both the noisy waveform $\epsilon_{\theta}(\,\cdot\,)$9 and the phase observation $x_t$0 are split into $x_t$1 non-overlapping patches of length $x_t$2, yielding two sequences of length $x_t$3 (Wang et al., 21 Mar 2026). RDT then adds separate learnable position embeddings to the two streams:

$x_t$4

Because these embeddings are independent, the model is described as enforcing strict one-to-one alignment: token $x_t$5 in the main stream can only attend, through the masked cross-attention mechanism, to tokens in the conditional stream that occupy corresponding or neighboring time windows (Wang et al., 21 Mar 2026). The stated consequence is prevention of drift in temporal correspondence.

This component is referred to as patch-level dual positional encodings. Its significance lies in the fact that the two sequences being related are not interchangeable modalities; one is a noisy estimate of the target respiration waveform, and the other is the radar phase observation conditioning the denoising process. Separate positional embeddings preserve the distinction between these streams while maintaining explicit temporal registration.

A plausible implication is that this mechanism addresses a common failure mode in cross-modal sequence modeling: semantically useful but temporally misaligned attention. In the RDT formulation, correspondence is not left entirely to learned global attention weights; it is constrained structurally through tokenization and positional design.

4. Banded-mask cross-attention and local physical prior

RDT injects a local physical prior by restricting cross-sequence attention to a temporal band of radius $x_t$6 (Wang et al., 21 Mar 2026). The mask matrix is defined as

$x_t$7

A single cross-attention head is then written as

$x_t$8

where $x_t$9 comes from the main tokens and $y$0 come from the conditional tokens (Wang et al., 21 Mar 2026). The banded mask ensures that each main token $y$1 only attends to conditional tokens $y$2 satisfying $y$3.

The stated rationale is physical: radar returns correlate locally in time with the true respiration, and the local attention band radius $y$4 encodes the prior that instantaneous chest motion most strongly influences temporally proximate radar echoes (Wang et al., 21 Mar 2026). This is a deliberately restrictive form of cross-attention. Rather than allowing arbitrary long-range token interactions across the two streams, the model privileges local temporal neighborhoods as the relevant support for conditioning.

This design is significant because it converts a domain assumption into a hard architectural constraint. In generic attention-based models, localness may emerge only implicitly. In RDT, locality is encoded directly through the mask. This suggests that the model is intended not only to learn a denoising map, but to do so under a constrained hypothesis class aligned with mmWave sensing physics.

5. Reverse diffusion procedure and computational profile

The reverse diffusion inference procedure uses a precomputed noise schedule $y$5 with $y$6, but selects 20 timesteps $y$7 via the “timestep respacing” strategy of Song et al. (Wang et al., 21 Mar 2026). Sampling is initialized as

$y$8

For $y$9, the predicted mean is computed as

$t$0

after which

$t$1

The final output is $t$2 after unfolding patches (Wang et al., 21 Mar 2026). The reverse predicted mean is also summarized as

$t$3

The framework states that, because sampling starts near $t$4 and only needs to remove the micromotion residual, 20 steps suffice to converge, in contrast to the 1,000+ steps of vanilla diffusion (Wang et al., 21 Mar 2026). In this formulation, RDT’s role in computational efficiency is inseparable from the observation-anchored initialization: the model is not tasked with generating a waveform from an uninformative prior, but with refining a measurement-consistent starting point.

A plausible implication is that the denoising burden placed on RDT is narrower than in unconstrained generative diffusion, which may help explain why a small Transformer-style architecture with $t$5 blocks is sufficient in the reported setting.

6. Empirical performance and ablation evidence

On held-out subjects, full mmWave-Diffusion with RDT achieves waveform reconstruction metrics of Cosine Similarity $t$6 and MSE $t$7, and respiration-rate metrics of MAE $t$8 BPM, RMSE $t$9 BPM, and SD $x_{t-1}$0 BPM (Wang et al., 21 Mar 2026). The best listed baseline, MM-FGRM, reports CS $x_{t-1}$1, MSE $x_{t-1}$2, and MAE $x_{t-1}$3 (Wang et al., 21 Mar 2026). The evaluation is conducted on 13.25 hours of synchronized radar-respiration data (Wang et al., 21 Mar 2026).

The ablation results isolate RDT’s architectural choices:

Variant	CS	MAE
No self-attention	0.753	1.274
No conditional tokens	0.758	1.062
Concatenate instead of cross-attention	0.762	0.957
Cross-attention without banded mask	0.790	0.804
Full (with banded cross-attention + dual encodings)	0.811	0.631

These results support several specific conclusions stated by the design. Removing self-attention degrades both waveform reconstruction and respiratory-rate estimation, indicating that intra-sequence dependency modeling in the main stream is important (Wang et al., 21 Mar 2026). Eliminating conditional tokens or replacing cross-attention with concatenation also reduces performance, which is consistent with the claim that explicit conditioning on radar phase observations is central to denoising (Wang et al., 21 Mar 2026). The comparison between unrestricted cross-attention and banded-mask cross-attention is especially informative: the full configuration improves from CS $x_{t-1}$4 and MAE $x_{t-1}$5 to CS $x_{t-1}$6 and MAE $x_{t-1}$7, indicating that the local physical prior embedded in the banded mask contributes materially to performance (Wang et al., 21 Mar 2026).

A plausible implication is that RDT’s gains do not derive from Transformer capacity alone, but from the conjunction of three constraints: explicit conditional tokens, dual positional encodings, and locality-restricted cross-attention.

7. Interpretation, scope, and relation to respiration sensing

RDT is presented as a component designed specifically for noisy, micromotion-corrupted radar phase sequences in contactless respiration sensing (Wang et al., 21 Mar 2026). Its defining characteristics are observation-anchored sampling, explicit conditioning on radar phase observations at every reverse step, strict one-to-one temporal alignment via patch-level dual positional encodings, and local physical priors through banded-mask multi-head cross-attention (Wang et al., 21 Mar 2026). Together, these choices are described as enabling robust denoising and interference removal in just 20 reverse steps.

Within this framing, RDT should not be understood as a generic Transformer attached to a diffusion model. Its architecture is specialized to a setting where the target signal and the conditioning observation are temporally synchronized, physically coupled, and only locally displaced by interference. The use of OCN initialization and local cross-attention implies a modeling assumption that the desired respiration waveform remains close, in a structured sense, to the observed radar phase sequence. This suggests that RDT is optimized for residual correction rather than unconstrained synthesis.

The principal misconception that the design counters is the idea that diffusion-based inference for physiological radar sensing must begin from pure noise and require very long reverse trajectories. In the reported formulation, the denoising process begins in an observation-consistent neighborhood and uses only 20 learned reverse steps (Wang et al., 21 Mar 2026). Another possible misconception is that global cross-attention should always be preferable in conditional sequence modeling. The ablation results indicate that, for this application, imposing a banded mask is beneficial (Wang et al., 21 Mar 2026).

In the context of mmWave respiration sensing, the significance of RDT lies in how it couples statistical denoising with a measurement-anchored inductive bias. The paper’s summary states that these design choices allow the model to exploit both the statistical power of diffusion models and the physics of mmWave breathing echoes, yielding rapid, high-fidelity reconstruction of respiratory waveforms from noisy observations (Wang et al., 21 Mar 2026).