USCTNet: Physics-Grounded HSI Reconstruction

• USCTNet is a physics-grounded deep unfolding network that formulates RGB-to-HSI recovery as a regularized inverse problem using a transform-domain nuclear norm prior.
• It employs a learnable proximal-gradient algorithm with a low-rank subspace SVT, maintaining colorimetric consistency and reducing computational costs.
• The model integrates estimated camera spectral sensitivity and illumination, outperforming baselines in PSNR, SSIM, and SAM for hyperspectral reconstruction.

Searching arXiv for the specified paper and closely related RGB-to-HSI reconstruction work to ground the article. USCTNet is a physics-grounded deep unfolding network for reconstructing a hyperspectral image (HSI) from a single RGB image. Introduced in "USCTNet: A deep unfolding nuclear-norm optimization solver for physically consistent HSI reconstruction" (Ma et al., 12 Sep 2025), it formulates RGB-to-HSI recovery as a regularized inverse problem under a camera and illumination-consistent forward model, explicitly estimates camera spectral sensitivity (CSS) and illumination, and solves the resulting problem via an unrolled proximal-gradient algorithm with a learnable transform-domain nuclear-norm prior. A central design objective is colorimetric consistency: the reconstructed HSI is constrained so that its RGB projection matches the measurement under the estimated imaging physics (Ma et al., 12 Sep 2025).

1. Problem setting and physical forward model

USCTNet addresses single-image RGB-to-HSI reconstruction, an ill-posed inverse problem because three RGB channels must be mapped to a substantially higher-dimensional spectral representation. The formulation used by the model is explicitly physics-grounded. For a pixel $p$ and channel $c \in \{R,G,B\}$, RGB formation is modeled as

$$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$

where $S_c(\lambda)$ is the CSS for channel $c$, $I(\lambda)$ is the spectral power distribution of illumination, $R(p,\lambda)$ is the spectral reflectance at pixel $p$, and $n_c$ is sensor noise (Ma et al., 12 Sep 2025).

After discretizing wavelengths on $B$ bands $c \in \{R,G,B\}$0, the forward operator is written in matrix form. Let $c \in \{R,G,B\}$1 collect spectral samples at $c \in \{R,G,B\}$2 pixels and $c \in \{R,G,B\}$3 denote the RGB image. Define $c \in \{R,G,B\}$4, $c \in \{R,G,B\}$5, and

$c \in \{R,G,B\}$6

The forward model becomes

$c \in \{R,G,B\}$7

In the paper’s notation, $c \in \{R,G,B\}$8 denotes the HSI and $c \in \{R,G,B\}$9 the RGB image, with forward relation

$$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$0

This matrix-tensor view preserves dimensions explicitly: $$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$1 maps the spectral dimension $$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$2 into 3 RGB channels, independently for each of the $$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$3 pixels. Embedding this operator into reconstruction is the mechanism by which USCTNet enforces physical consistency rather than relying only on statistical RGB-to-HSI regression.

2. Inverse formulation and transform-domain nuclear norm

The reconstruction objective is posed as a regularized inverse problem in a learnable transform domain. USCTNet solves

$$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$4

where $$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$5 is the nuclear norm, $$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$6 is a learnable analysis operator, and $$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$7 is its synthesis operator (Ma et al., 12 Sep 2025).

The prior acts primarily along the spectral dimension. The nuclear norm in the transform domain promotes low-rank structure in $$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$8, which is well-suited to HSIs because of strong spectral correlations and low intrinsic spectral dimensionality. This is not a generic low-rank prior on the image itself, but a structural prior in a learned analysis space. The data-fidelity term uses the estimated $$y_c(p) = \int S_c(\lambda) I(\lambda) R(p,\lambda)\, d\lambda + n_c(p),$$9, so the optimization directly couples spectral reconstruction to CSS and illumination. The paper states that this mitigates color mismatches by providing a physically consistent mapping between reconstructed HSI and measured RGB (Ma et al., 12 Sep 2025).

This formulation places USCTNet at the intersection of model-based inverse problems and learned priors. The inverse problem remains explicit, while the prior and proximal updates are learned. A plausible implication is that the method inherits some of the interpretability of optimization-based reconstruction while preserving the representational capacity of deep networks.

3. Deep unfolding architecture

USCTNet unrolls a proximal-gradient solver for $S_c(\lambda)$0 stages. A parameter-estimation module, PhiNet, predicts a forward operator $S_c(\lambda)$1 from the input RGB and is shared across all stages. Each stage performs the following updates (Ma et al., 12 Sep 2025):

$S_c(\lambda)$2

$S_c(\lambda)$3

$S_c(\lambda)$4

$S_c(\lambda)$5

$S_c(\lambda)$6

The compact form is

$S_c(\lambda)$7

Here, $S_c(\lambda)$8 and $S_c(\lambda)$9 are learnable per-stage step sizes and thresholds, while the transform $c$0 and synthesis $c$1 are shared across stages. The update $c$2 is described as a color-consistency force aligned with imaging physics, because it constrains the RGB projection $c$3 to match the measurement $c$4 (Ma et al., 12 Sep 2025).

The backbone couples this iterative structure to a transformer-based encoder-decoder adapted from MST++ to strengthen cross-band representation. Convolution-guided channel attention (CAB) and dual residual transformer blocks (DRTBs) are added to stabilize high-dimensional features. Together with the low-rank subspace proximal operator, these form the proximal block, denoted LRSPB. A memory module $c$5 provides cross-stage spectral context (Ma et al., 12 Sep 2025).

PhiNet is pre-trained on the training split and frozen during backbone training. The stated purpose is optimization stability: $c$6 is estimated once, then embedded consistently across all unfolding stages.

4. Low-rank subspace SVT proximal operator

A key contribution of USCTNet is its replacement of full singular-value decompositions with a data-adaptive low-rank subspace SVT proximal operator. Standard singular-value thresholding for $c$7 with thin SVD $c$8 is

$c$9

Repeated full SVDs are costly and unstable for high-dimensional features. USCTNet instead applies a subspace SVT variant tailored to analysis-domain features $I(\lambda)$0 (Ma et al., 12 Sep 2025).

The operator has three components. First, differentiable subspace selection chooses a small set of informative columns via gated scoring and a soft top-$I(\lambda)$1 operator, forming a sketch $I(\lambda)$2 and an orthonormal basis

$I(\lambda)$3

with $I(\lambda)$4. Second, adaptive probing and threshold scheduling estimate a residual energy ratio using random probes $I(\lambda)$5,

$I(\lambda)$6

and a lightweight MLP predicts a threshold increment $I(\lambda)$7 and refined residual $I(\lambda)$8. Third, the proximal is applied in the selected subspace. USCTNet computes a thin SVD of $I(\lambda)$9, performs soft-shrinkage on the singular values, and maps the result back to the ambient space. Multiple proposals are fused with residual-aware weights $R(p,\lambda)$0 (Ma et al., 12 Sep 2025).

The complexity reduction is explicit. Full SVD on $R(p,\lambda)$1 costs $R(p,\lambda)$2, whereas the subspace method costs $R(p,\lambda)$3 per inner step with $R(p,\lambda)$4. The paper attributes substantial savings and improved gradient stability to avoiding full-spectrum singular-value operations (Ma et al., 12 Sep 2025).

Within the overall solver, LRSP is the learned proximal realization of the transform-domain nuclear norm. The ablation studies emphasize that replacing full SVT with LRSP yields the largest improvement by controlling effective rank and smoothing gradients, whereas learnable GD+prox alone provides limited gains due to instability with full SVT.

5. Training protocol, datasets, and empirical performance

USCTNet is evaluated on ARAD-1K Real and ICVL. Because ICVL lacks paired RGB, RGBs are synthesized following prior work. Eighteen inconsistent-resolution images are removed, yielding 153 training and 30 test pairs for ICVL (Ma et al., 12 Sep 2025).

Training uses end-to-end optimization with $R(p,\lambda)$5 reconstruction loss on HSIs. The optimizer is Adam with learning rate $R(p,\lambda)$6, $R(p,\lambda)$7, $R(p,\lambda)$8, cosine annealing, 300 epochs, and batch size 8. Inputs are normalized to $R(p,\lambda)$9; training uses $p$0 patches with stride 8 and augmentation via random flips and rotations. Experiments are performed on NVIDIA A100 (40 GB) (Ma et al., 12 Sep 2025).

Ablation indicates that $p$1 unfolding stages balance convergence and complexity: $p$2 underconverges, while $p$3 overfits. The reported model size is approximately $p$4M parameters with approximately $p$5G FLOPs. For deployment on a new camera, the recommended procedure is to estimate CSS $p$6 via calibration or manufacturer data, estimate illumination $p$7 via spectral measurement or scene-adaptive estimation, pre-train or fine-tune PhiNet to predict $p$8 accurately, and run USCTNet with $p$9 stages (Ma et al., 12 Sep 2025).

The benchmark results reported for reconstruction accuracy are as follows:

Method	ARAD-1K Real (PSNR / SSIM / SAM)	ICVL (PSNR / SSIM / SAM)
USCTNet	36.56 / 0.941 / 4.69	35.13 / 0.979 / 2.79
MST++	34.36 / 0.927 / 5.10	31.24 / 0.968 / 3.05
CESST	35.19 / 0.932 / 5.73	33.16 / 0.978 / 2.64

USCTNet is reported to consistently improve PSNR and SSIM while lowering SAM relative to these recent baselines. The paper also reports RGB reproduction metrics including PSNR, SSIM, and $n_c$0, with lower color differences and improved RGB fidelity; figure-level examples indicate close RGB matches to ground truth (Ma et al., 12 Sep 2025).

Code is available at https://github.com/psykheXX/USCTNet-Code-Implementation.git, and the released configuration matches the reported parameter count and FLOPs (Ma et al., 12 Sep 2025).

6. Robustness, limitations, and nomenclature

The explicit estimation and embedding of $n_c$1 is presented as the main mechanism for robustness under camera and illumination shifts. By driving gradient descent with $n_c$2, the network maintains colorimetric consistency even when CSS or illumination differ across scenes. Evaluations include $n_c$3, indicating improved color fidelity (Ma et al., 12 Sep 2025).

The method nevertheless retains several assumptions and limitations. It is sensitive to $n_c$4 estimation: errors in CSS or illumination estimation propagate to the data-fidelity term and can affect both color reproduction and spectra. The architecture is also heavier than purely feed-forward networks such as MST++, because unfolding with physics-guided gradient steps and subspace SVT incurs additional overhead, even though LRSP substantially reduces cost relative to full SVT. Large changes in camera or illumination spectra may still degrade performance, and re-training or fine-tuning PhiNet is recommended under domain shift (Ma et al., 12 Sep 2025).

A likely misconception concerns the acronym itself. In (Ma et al., 12 Sep 2025), USCTNet denotes a deep unfolding solver for physically consistent RGB-to-HSI reconstruction. This should not be conflated with the use of “USCTNet” as a conceptual label for learned inversion in ultrasound computed tomography, where a CNN-based full-waveform learned inversion maps USCT waveform data to speed-of-sound images (Lozenski et al., 2023). The shared acronym therefore spans distinct technical domains: hyperspectral reconstruction in one case, ultrasound computed tomography in the other.

Within RGB-to-HSI reconstruction, USCTNet is positioned relative to prior CNN and Transformer methods that rely on spatio-spectral correlations without physics, and relative to unfolding approaches that retain full SVT. Its specific contribution is the combination of a physics-grounded operator with learned CSS and illumination, a transform-domain nuclear norm, and a differentiable full-SVD-free subspace SVT proximal operator. This suggests that its distinctive identity lies less in any single architectural block than in the coupling of physically accurate data fidelity with a computationally tractable low-rank proximal mechanism (Ma et al., 12 Sep 2025).