Flow-Matching Unfolding Network (FMU)

• The paper introduces a novel integration of flow matching into a deep unfolding framework to recover hyperspectral image cubes from compressed and highly ill-posed measurements.
• The method employs a two-phase training strategy combining a latent encoder with a mean velocity loss to enforce global consistency in the learned flow field.
• Experimental results demonstrate that FMU outperforms CNN, RNN, and Transformer baselines, achieving higher PSNR and SSIM on both simulated and real hyperspectral data.

Searching arXiv for the primary FMU paper and closely related unfolding/flow-matching work to ground the article in current literature. Flow-Matching-guided Unfolding network (FMU) is a hyperspectral image reconstruction architecture for compressed sensing systems that embeds a flow-matching generative prior within a deep unfolding framework. It is designed for the recovery of a hyperspectral image (HSI) cube from compressed measurements produced by optical-filter systems or coded aperture snapshot spectral imaging (CASSI), where the inverse problem is highly ill-posed and fine spectral details are easily lost. FMU combines a Generalized Alternating Projection (GAP) data-consistency update with a prior-guided denoising stage, and introduces a mean velocity loss to enforce global consistency of the learned flow field. In the source formulation, this is presented as, to the authors’ knowledge, the first integration of flow matching into HSI reconstruction, with the stated aim of combining optimization interpretability and generative prior modeling (Ai et al., 2 Oct 2025).

1. Measurement model and inverse problem

FMU is formulated for an HSI cube

$$\mathbf{X} \in \mathbb{R}^{W \times H \times N_\lambda},$$

where $W$ and $H$ are spatial dimensions and $N_\lambda$ is the number of spectral channels. The measurement depends on the sensing hardware.

For optical-filter systems, the measurement is a two-dimensional image

$\mathbf{Y} \in \mathbb{R}^{W \times H},$

generated by a multiplexed spectral encoding

$$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$

with $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$ a three-dimensional mask and $\mathbf{N}$ additive noise. This design increases spectral multiplexing and reconstruction difficulty.

For CASSI, the measurement has shifted spatial extent,

$$\mathbf{Y}_{\text{CASSI}} \in \mathbb{R}^{W \times (H + (N_\lambda - 1)d)},$$

because a two-dimensional mask $\mathbf{m}$ is shifted along the spectral direction with step $W$0. The corresponding model is

$W$1

with

$W$2

Both systems are unified in a vectorized linear model. Let

$W$3

and let $W$4 denote vectorized noise. With sensing matrix

$W$5

the forward model is

$W$6

The reconstruction target is therefore $W$7 from $W$8 under a severely underdetermined linear inverse problem. This formulation places FMU within the standard compressed sensing view of HSI reconstruction, while preserving explicit dependence on the sensing operator.

2. Unfolding formulation and optimization interpretation

FMU adopts a deep unfolding strategy derived from an optimization problem of the form

$W$9

where the first term enforces measurement fidelity and $H$0 is a regularizer weighted by $H$1.

To decouple fidelity and prior terms, the method introduces an auxiliary variable $H$2 and rewrites the problem as

$H$3

Under this interpretation, $H$4 is the measurement-consistent variable, and $H$5 is the prior-refined variable.

The unfolded iteration consists of two steps at each stage. The projection step enforces data fidelity: $H$6 where $H$7 is the pseudo-inverse or back-projection operator. The denoising step applies a learned prior: $H$8 where $H$9 is the stage-wise denoiser and $N_\lambda$0 is the flow-matching prior.

This stage decomposition is central to the interpretability claim of FMU. Each unfolding layer corresponds to a recognizable optimization step: projection onto the measurement constraint followed by prior-based regularization. The denoiser is therefore not an unconstrained post-processing module, but the learned analogue of the regularization or prox step in the auxiliary-variable formulation.

3. Flow-matching prior and mean velocity constraint

The distinctive element of FMU is the replacement of a conventional learned prior by a flow-matching latent prior. Flow matching is used as a conditional generative model in latent space, defined by a deterministic ordinary differential equation

$N_\lambda$1

In the stated construction, $N_\lambda$2 is drawn from latent features extracted from clean HSIs and $N_\lambda$3 from a standard Gaussian. The path between them follows linear interpolation,

$N_\lambda$4

which yields a constant target velocity

$N_\lambda$5

The basic regression objective is

$N_\lambda$6

Relative to diffusion-style constructions, the description emphasizes direct velocity regression on continuous-time paths, path independence, and deterministic ODE sampling with fewer steps.

The prior used by the unfolding network is generated in latent space. A latent encoder LE maps the ground-truth HSI $N_\lambda$7 and normalized measurement

$N_\lambda$8

to a latent prior $N_\lambda$9. The encoder input is

$\mathbf{Y} \in \mathbb{R}^{W \times H},$0

and its architecture uses pixel unshuffle, MobileBlocks, and MLP-Mixer components. In the second training phase, flow matching learns to generate $\mathbf{Y} \in \mathbb{R}^{W \times H},$1 conditioned on $\mathbf{Y} \in \mathbb{R}^{W \times H},$2, with $\mathbf{Y} \in \mathbb{R}^{W \times H},$3. This generated sample is the prior $\mathbf{Y} \in \mathbb{R}^{W \times H},$4 passed to the denoiser (Ai et al., 2 Oct 2025).

FMU supplements the standard flow-matching objective with a mean velocity loss,

$\mathbf{Y} \in \mathbb{R}^{W \times H},$5

The stated purpose is to improve global consistency of the learned velocity field and stabilize the flow. The final flow loss is

$\mathbf{Y} \in \mathbb{R}^{W \times H},$6

The article’s formulation makes a specific conceptual distinction: the flow prior is not a closed-form proximal operator. Instead, it is a strong generative prior learned from clean HSIs and injected into the denoiser as latent guidance.

4. Architecture and training procedure

The full FMU pipeline begins from a measurement $\mathbf{Y} \in \mathbb{R}^{W \times H},$7 and sensing matrix $\mathbf{Y} \in \mathbb{R}^{W \times H},$8. A back-projected estimate

$\mathbf{Y} \in \mathbb{R}^{W \times H},$9

is computed first. The trained flow module then generates

$$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$0

conditioned on $$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$1. This latent prior is reused by an $$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$2-stage unfolding network, where each stage applies the GAP projection update and a denoising block conditioned on $$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$3.

The denoiser $$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$4 is a U-shaped Transformer with Trident Transformer modules. In the source description, these modules aggregate degradation-free latent prior features from $$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$5 and fuse them with the current unfolded estimate. The U-shaped structure, together with attention-based fusion, is intended to exploit spectral-spatial dependencies while injecting prior information at each unfolding stage.

The flow velocity network itself is implemented as a denoiser-like network. The reported ablation considers MLP, gMLP, Tiny Transformer, and SimpleCNN variants, with SimpleCNN giving the best trade-off. The flow integration is described as relatively lightweight because it is performed in latent space rather than image space.

Training is organized into two phases. In Phase 1, the latent encoder and unfolding network are trained using the encoder-produced prior: $$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$6 The flow module is not yet used.

In Phase 2, the latent encoder is frozen. Flow matching learns to generate $$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$7 conditioned on $$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$8, reconstruction is performed with that generated prior,

$$\mathbf{Y} = \sum_{n_\lambda=1}^{N_\lambda}\mathbf{X}(:,:,n_\lambda)\odot \mathbf{M}(:,:,n_\lambda) + \mathbf{N},$$9

and the total loss becomes

$$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$0

The implementation details explicitly reported include 28 spectral channels spanning 450–650 nm, 300 training epochs, Adam with $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$1, and cosine annealing from $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$2 to $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$3. FMU is reported at 4.09M parameters and 98.84 G FLOPs for $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$4 inputs, while FMU-S uses 2.78M parameters and 99.87 G FLOPs. The optimal mean-velocity weight is reported as $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$5.

5. Experimental evaluation and ablation evidence

The simulated optical-filter evaluation uses the CAVE dataset for training, with 32 HSIs at $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$6, and the KAIST dataset for testing, with 30 HSIs at $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$7; 10 representative scenes are used for evaluation. Measurements are simulated with optical filter-based masks described as Fabry–Perot filters. Real-data evaluation uses the TSA-Net CASSI setup, with training samples generated as simulated CASSI measurements with noise consistent with the real optical setup. Performance is reported with PSNR and SSIM (Ai et al., 2 Oct 2025).

The main simulated results position FMU above the listed CNN-based, RNN-based, Transformer-based, and unfolding-based baselines. Average performance over the evaluated scenes is summarized below.

Model	Average PSNR / SSIM	Parameters / FLOPs
FMU	42.13 dB / 0.9900	4.09M / 98.84G
LADE-DUN	40.97 dB / 0.9882	2.78M / 96.69G
FMU-S	41.94 dB / 0.9894	2.78M / 99.87G

The broader unfolding comparison reported for the same setting lists ADMM-Net at 34.93 dB / 0.9570, GAP-Net at 36.14 dB / 0.9653, DAUHST at 38.81 dB / 0.9815, LADE-DUN at 40.97 dB / 0.9882, and FMU at 42.13 dB / 0.9900. Within the ablation on prior modeling, the baseline without prior gives 40.58 dB / 0.9878 at 96.40 G FLOPs, adding a latent diffusion prior gives 40.97 dB / 0.9882 at 96.69 G FLOPs, and adding flow matching gives 42.13 dB / 0.9900 at 98.84 G FLOPs. This isolates the contribution of the flow-matching prior under nearly unchanged computational scale.

The mean-velocity ablation identifies $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$8 as the best setting, yielding 42.13 dB and 0.9900 SSIM; smaller or larger values degrade performance. The velocity-network ablation reports 41.95 dB for MLP, 42.05 dB for gMLP, 41.90 dB for Tiny Transformer, and 42.13 dB for SimpleCNN, with SimpleCNN also giving the lowest FLOPs among the stronger variants.

Qualitatively, the simulated comparison examines four channels around 481.5, 522.5, 575.5, and 648.0 nm. FMU is described as producing sharper edges, more detailed textures, and better recovery of high-frequency structures. Spectral density curves reportedly show the highest correlation with ground truth, at 0.9417. On real CASSI measurements, where no ground truth is available, the reported visual outcome is clearer spatial detail, fewer artifacts, better cross-channel consistency, and smoother spectral variation.

6. Position within the literature, misconceptions, and limitations

FMU belongs to a broader convergence between generative modeling and algorithm unrolling, but it should not be conflated with other “flow-inspired” constructions. In general flow-matching literature, the core object is a deterministic ODE that transports a base distribution to a target distribution through a learned velocity field, often under a linear interpolation path with constant target velocity; this is the formal background to FMU’s latent prior model, rather than a property unique to hyperspectral reconstruction (Lipman et al., 2024).

Within spectral imaging, a closely related but distinct line is represented by "Progressive Flow-inspired Unfolding for Spectral Compressive Imaging" (Wang et al., 15 Sep 2025). That method, named FLoUNet, uses trajectory-controllable unfolding with a learned convex interpolation and a trajectory loss to enforce smooth stage-wise refinement. FMU differs in the specific mechanism by which flow concepts enter the reconstruction: it embeds a latent flow-matching generative prior inside a GAP-based unfolding network and adds a mean velocity loss, whereas FLoUNet constrains the unfolding trajectory itself through interpolation weights and trajectory supervision. The two methods are therefore related in motivation but not identical in architecture or optimization semantics.

A further possible misconception is to generalize the term “FMU” to any few-step or guidance-based FM system. The speech-generation work "Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesis" explicitly states that it does not use the term FMU; it discusses data-guidance, intrinsic guidance distillation, and trajectory rectification for speech synthesis rather than inverse reconstruction (Yu et al., 1 Jul 2026). Its relevance is conceptual: it shows that flow guidance can be integrated with staged generation and efficiency-oriented design, but it does not define the hyperspectral reconstruction architecture called FMU.

The source discussion does not provide an extensive limitations section, but several implications are explicitly suggested. A plausible implication is that the computational burden remains nontrivial, at roughly 99 G FLOPs for $$\mathbf{M} \in \mathbb{R}^{W \times H \times N_\lambda}$$9 inputs, even if the flow-prior overhead over strong unfolding baselines is small. A second implication is that the two-phase training pipeline, with encoder pretraining followed by joint flow and reconstruction training, is more complex than single-stage regression. A third implication is that the method remains tied to the sensing matrix $\mathbf{N}$0, so deployment on different hardware may require retraining or careful forward-model adaptation.

The future directions hinted in the description are correspondingly hardware- and modality-oriented: deployment in compact chip-integrated HSI devices, extension to other computational imaging modalities such as CT, MRI, coded aperture imaging, and lensless imaging, and exploration of more advanced flow variants such as Rectified Flow or FM-OT. These suggestions are consistent with the underlying hybrid design of FMU, in which measurement physics and a learned generative prior are combined rather than treated as competing paradigms.