MultiCE-Flow: Multimodal Channel Estimation

• MultiCE-Flow is a multimodal deep generative framework that fuses LiDAR, camera, and location sensor inputs with sparse pilot measurements to accurately reconstruct MIMO-OFDM channels.
• It leverages a diffusion transformer backbone with flow matching for rapid and sample-efficient one-step inference, achieving substantial NMSE improvements over classical methods.
• The integration of environmental semantics via classifier-free guidance enables robust channel estimation under severe pilot scarcity and out-of-distribution conditions.

MultiCE-Flow is a multimodal deep generative framework for channel estimation in sensing-aided wireless communication systems, employing flow matching and a diffusion transformer (DiT) backbone. By fusing LiDAR, camera, and location sensor data with sparse pilot measurements, MultiCE-Flow leverages environmental semantics to reconstruct frequency-domain MIMO-OFDM channels, achieving robust and sample-efficient inference. Efficient single-step generation via flow matching distinguishes MultiCE-Flow from standard diffusion models and enables real-time operation under severe pilot scarcity and out-of-distribution environmental conditions (Fan et al., 13 Mar 2026).

1. Channel Estimation Problem and Data Modalities

MultiCE-Flow is formulated for downlink vehicle-to-infrastructure (V2I) MIMO-OFDM with a roadside unit (RSU) transmitting to a connected and autonomous vehicle (CAV). The system operates with $N_t$ transmit and $N_r$ receive antennas across $N_c$ subcarriers. At each subcarrier $k$, the frequency-domain channel is

$$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$

where $\tau_l$ denotes multipath delays.

A subset $\mathcal{P}\subset\{0,\dots,N_c-1\}$ is selected for pilot transmission; at each $k\in\mathcal{P}$, only antenna $t_k$ transmits a symbol $s[k]$, producing the observation

$N_r$0

The objective is to recover the full channel tensor $N_r$1 from the sparse pilot set $N_r$2 and auxiliary multimodal data.

MultiCE-Flow employs:

• LiDAR point clouds: Processed into bird’s-eye-view (BEV) semantic maps.
• Multi-view camera images: Four RGB cameras, ResNet-34 encoders.
• Fine-grained location information: Relative $N_r$3 vectors.

2. Multimodal Perception and Conditioning

The model employs parallel branches for each input type:

• LiDAR: Converts BEV maps to $N_r$4 tokens via a perceiver resampler.
• Camera: Extracts per-view features to $N_r$5 tokens, incorporating view-ID embeddings.
• Location: Embeds relative coordinates as $N_r$6 tokens using an MLP.

Tokens from all branches are concatenated,

$N_r$7

then enriched with modality/positional embeddings and fused through a multi-layer transformer encoder to produce semantic environmental condition $N_r$8.

Pilot-based observations are processed into a "structural condition" $N_r$9:

1. Least-squares (LS) solution with pilot interpolation yields $N_c$0.
2. Transformation to angle-delay domain (2-D DFT); real and imaginary components split and stacked to obtain $N_c$1.

3. Diffusion Transformer and Flow Matching

Both the ground-truth channel $N_c$2 and a Gaussian noise initialization $N_c$3 are angle-delay transformed and tokenized via 3D patching and projection. An interpolation trajectory parameterized by $N_c$4 is constructed:

$N_c$5

Tokens of $N_c$6 and $N_c$7 are concatenated, positional embeddings are added, and input is provided to a DiT backbone. Each DiT block features:

• Cross-attention to $N_c$8
• Timestep embedding via adaptive layer normalization

The core innovation is the adoption of a flow matching objective. The true velocity field is straightforward:

$N_c$9

The network $k$0 is trained to regress this straight-line velocity:

$k$1

This allows deterministic, efficient one-step inference.

4. Training and Inference Pipeline

Training: For each minibatch:

• Sample $k$2, $k$3, form $k$4.
• Construct $k$5 and $k$6.
• Tokenize and input to DiT.
• Regress velocity; perform weight updates via AdamW.

Inference: Given observed pilots and environmental data,

• Form $k$7, $k$8.
• Sample noise $k$9.
• Compute guided velocity using classifier-free guidance (CFG):

$$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$0

• Output channel estimate via one Euler step:

$$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$1

• Inverse transform to frequency domain.

CFG enables robust, adjustable semantic conditioning by randomly dropping $$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$2 at training, enabling both conditional and unconditional guidance at inference.

5. Experimental Evaluation

MultiCE-Flow is evaluated on the Multimodal-Wireless dataset: 16 scenarios, 640k training samples, 80k in-distribution test samples, and a 48k out-of-distribution (OOD) test set (e.g., rural “Town07”).

Baselines: LS, LMMSE, DMPS (iterative DiT posterior), and Pilots-Only (MultiCE-Flow with $$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$3).

Results:

• At $$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$4 dB SNR (pilot spacing $$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$5):
  • LS: $$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$6 dB NMSE
  • LMMSE: $$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$7 dB
  • DMPS: $$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$8 dB
  • Pilots-Only: $$\mathbf{H}[k] = \sum_{l=1}^L \mathbf{A}_l e^{-j2\pi k \Delta f \tau_l},\quad \mathbf{A}_l \in \mathbb{C}^{N_r\times N_t}$$9 dB
  • MultiCE-Flow: $\tau_l$0 dB
• At $\tau_l$1, MultiCE-Flow ($\tau_l$2 dB NMSE) outperforms LMMSE at $\tau_l$3 ($\tau_l$4 dB).
• Environmental ablation ($\tau_l$5, SNR $\tau_l$6 dB, NMSE): Pilots-Only ($\tau_l$7 dB), +Location ($\tau_l$8 dB), +LiDAR ($\tau_l$9 dB), +Camera ($\mathcal{P}\subset\{0,\dots,N_c-1\}$0 dB), Full ($\mathcal{P}\subset\{0,\dots,N_c-1\}$1 dB).
• OOD: At $\mathcal{P}\subset\{0,\dots,N_c-1\}$2 dB SNR, MultiCE-Flow exceeds Pilots-Only by $\mathcal{P}\subset\{0,\dots,N_c-1\}$3 dB, cosine similarity $\mathcal{P}\subset\{0,\dots,N_c-1\}$4 with $\mathcal{P}\subset\{0,\dots,N_c-1\}$5.
• Latency per sample: LMMSE $\mathcal{P}\subset\{0,\dots,N_c-1\}$6 ms, DMPS $\mathcal{P}\subset\{0,\dots,N_c-1\}$7 ms (69.4 GFLOPs), MultiCE-Flow $\mathcal{P}\subset\{0,\dots,N_c-1\}$8 ms (5.6 GFLOPs).

6. Comparison with Classical and Generative Estimators

Classical estimators perform poorly under low SNR and pilot scarcity; LS is unreliable, while LMMSE requires prior covariance knowledge and assumes linearity. MultiCE-Flow demonstrates substantial NMSE gains (5–10 dB) in these regimes.

Relative to other deep generative approaches:

• DMPS’s iterative DiT posterior sampling incurs higher latency and sensitivity to noise.
• MultiCE-Flow’s flow-matching enables single-step, deterministic mapping, greatly reducing latency while retaining semantic controllability via CFG.

Integration of environmental context mitigates the ill-posedness caused by pilot sparsity; single-step inference accommodates real-time constraints; CFG scaling enhances OOD robustness.

7. Limitations and Prospects

MultiCE-Flow presumes high-quality, synchronized multimodal sensor inputs and accurate cross-modal alignment. The large DiT backbone imposes significant training cost and memory requirements. Applicability to scenarios with missing or unreliable environmental modalities may require augmentation.

A plausible implication is that future research may investigate lightweight, adaptive backbones or self-supervised cross-modal alignment to reduce sensor and computation dependencies. Extending MultiCE-Flow beyond V2I scenarios, or integrating temporal context, may provide additional generalizability across dynamic communication regimes (Fan et al., 13 Mar 2026).