HS-SISR: Hyperspectral Image Super-Resolution

• HS-SISR is the task of enhancing hyperspectral images by recovering high-resolution spectral or spatial details from a single low-resolution observation.
• It employs supervised deep architectures, implicit neural representations, and synthetic-data training to address challenges like ill-posedness and data scarcity.
• Recent advances integrate physically constrained models and meta-learning techniques to adapt to sensor variability and maintain spectral fidelity.

Hyperspectral Single Image Super-Resolution (HS-SISR) is a class of inverse problems focused on enhancing either the spatial or spectral resolution of hyperspectral images using only a single low-resolution (LR) observation. Unlike classical super-resolution, which typically targets spatial enhancement, HS-SISR encompasses problems where the goal is to recover a hyperspectral image (HSI) from degraded spectral or spatial observations. This task arises in remote sensing, material analysis, and scientific imaging, where sensor hardware typically entails a trade-off between spatial, spectral, and temporal resolution. Modern HS-SISR leverages a range of techniques, including deep convolutional networks, meta-learning, implicit neural representations, and unsupervised synthetic-data strategies, to address challenges posed by ill-posedness, data scarcity, and sensor variability.

1. Mathematical Models for HS-SISR

HS-SISR problems differ by their degradation model and the nature of the available observation. The classical spectral SISR formulation consists of predicting a high-spectral-resolution image $Y \in \mathbb{R}^{H \times W \times B}$ from an input $X \in \mathbb{R}^{H \times W \times 3}$ (e.g., an RGB image):

$Y \approx f_\theta(X)$

where $f_\theta$ denotes a parameterized mapping (typically a neural network) (Galliani et al., 2017).

For spatial SISR, the model assumes a spatially downsampled observation, often described by:

$Y = D(B(X)) + N$

where $D$ is a downsampling operator, $B$ is a spatial blur, $N$ denotes noise, and $Y$ is the LR-HSI to be super-resolved to an HR-HSI $X$ (Muhammad et al., 6 May 2025).

Hybrid models, particularly in remote sensing, consider both low-resolution hyperspectral ($X \in \mathbb{R}^{H \times W \times 3}$0) and high-resolution multispectral (e.g. RGB) images ($X \in \mathbb{R}^{H \times W \times 3}$1), linked by:

$X \in \mathbb{R}^{H \times W \times 3}$2

where $X \in \mathbb{R}^{H \times W \times 3}$3 is the latent HR-HSI, $X \in \mathbb{R}^{H \times W \times 3}$4 is the spectral response function, and $X \in \mathbb{R}^{H \times W \times 3}$5 the downsampling operator (Li et al., 2024).

Physically-constrained or unmixing-based models decompose the HSI into endmembers and abundances:

$X \in \mathbb{R}^{H \times W \times 3}$6

where $X \in \mathbb{R}^{H \times W \times 3}$7 are spectral endmembers and $X \in \mathbb{R}^{H \times W \times 3}$8 the abundance maps (Xu et al., 30 Jan 2026, Xu et al., 30 Jan 2026, Xu et al., 23 Jan 2026).

2. Supervised Deep Architectures and Implicit Representation

A dominant approach in HS-SISR is supervised deep learning. Early works employed encoder-decoder ConvNets to map RGB to HSI with fixed or variable output bands. The “Tiramisu” CNN with DenseNet skip connections and subpixel upsampling learns an end-to-end RGB-to-HS mapping:

• Input: $X \in \mathbb{R}^{H \times W \times 3}$9
• Output: $Y \approx f_\theta(X)$0
• Architecture: DenseNet blocks, max pooling, subpixel (“pixel shuffle”) layers, skip connections
• Losses: MSE, optionally Spectral Angle Mapper (SAM) loss (Galliani et al., 2017).

Multi-scale U-Net style CNNs explicitly aggregate information from multiple resolution levels via symmetric downsampling and upsampling, with skip connections to preserve detail. Only per-pixel MSE is used for training, although evaluation may involve SAM and RMSE (Yan et al., 2018).

Recent advances include:

• Inception-style blocks to capture multi-scale spatial dependencies (Muhammad et al., 6 May 2025)
• Dual-domain networks leveraging both spatial convolution (Spatial-Net) and wavelet-domain branches (DWT) to separate smooth and textural details (Karayaka et al., 10 Dec 2025)
• Hybrid “unmixing” modules that integrate explicit or learned spectral decomposition into spatial–spectral CNN backbones (Muhammad et al., 26 Sep 2025)

Implicit neural representation (INR) methods frame HS-SISR as learning a continuous function $Y \approx f_\theta(X)$1 that regresses a high-resolution spectral vector at each spatial coordinate. A hypernetwork predicts the weights of per-patch or per-cell MLPs, enabling content-adaptive, continuous recovery. Periodic coordinate encodings boost high-frequency accuracy (Zhang, 2021).

3. Learning Paradigms: Meta-Learning, Data Augmentation, and Transfer

Meta-learning addresses sensor diversity by conditioning the network on spectral/physical metadata:

• MLSR employs hypernetworks (“W2WNet”) to produce convolution weights as a function of input and output band wavelengths, enabling a single model to handle arbitrary band settings (Zhang et al., 2021).

Data scarcity is mitigated via:

• Spectral Mixup—virtual samples are generated by random band mixing to increase spectral diversity and improve generalization (Li et al., 2021, Li et al., 2024).
• Multi-task learning—joint training on RGB-SISR and HS-SISR branches shares an encoder, providing stronger spatial-spectral priors and enabling semi-supervised extension to unlabelled data (Li et al., 2021).

Recent transfer-based frameworks such as EigenSR map the spectral dimension to a low-rank eigenimage basis. Pre-trained RGB super-resolution models are adapted to enhance each eigenimage (spatial mode), while Iterative Spectral Regularization ensures the upsampled result remains consistent with the low-dimensional spectral manifold (Su et al., 2024).

4. Unsupervised and Synthetic-Data Training Strategies

The scarcity of ground-truth HR-HSI motivates unsupervised pipelines. A dominant paradigm decomposes LR-HSI into endmembers and abundances (“unmixing”), then super-resolves abundances via a deep network trained on synthetic data generated by the dead leaves model—a spatial process that produces synthetic abundance patches with realistic geometric and marginal statistics (Xu et al., 23 Jan 2026, Xu et al., 30 Jan 2026, Xu et al., 30 Jan 2026).

The typical pipeline is:

1. Unmix LR-HSI to obtain abundance maps $Y \approx f_\theta(X)$2 and endmembers $Y \approx f_\theta(X)$3.
2. Generate synthetic HR–LR abundance pairs $Y \approx f_\theta(X)$4 via dead leaves and physical degradation (PSF, downsampling).
3. Train an abundance super-resolution network (e.g., MCNet, RDN) solely on synthetic data.
4. At inference, super-resolve $Y \approx f_\theta(X)$5 to obtain $Y \approx f_\theta(X)$6, then reconstruct HR-HSI by $Y \approx f_\theta(X)$7.

Noise-aware variants inject realistic noise into synthetic maps to enhance robustness. Results surpass classical and some supervised baselines on typical benchmarks (Urban, PaviaU, Chikusei) (Xu et al., 30 Jan 2026, Xu et al., 30 Jan 2026).

5. Evaluation Benchmarks, Losses, and Quantitative Results

Common datasets:

• CAVE, ICVL, NUS for laboratory HSIs (31 bands, 400–700nm)
• PaviaU, PaviaC, Chikusei for remote sensing (102–128 bands)
• NTIRE2018/2020 for spatial and spectral scaling, with train/test splits

Main metrics:

• RMSE, PSNR—spatial-spectral fidelity
• SAM—spectral angle error
• ERGAS—normalized global error
• MSSIM/SSIM—structural similarity

Summary table: | Method | Dataset | Notable Results | |-----------------|----------------|------------------------------------------| | Tiramisu-CNN | ICVL | RMSE=1.98, SAM=2.04°, SOTA (Galliani et al., 2017) | | FGIN | PaviaC 2× | MPSNR=36.57dB, MSSIM=0.9570, SAM=3.74° | | DDSRNet | PaviaU 4× | MPSNR=30.56dB, MSSIM=0.8181, SAM=4.84° | | EigenSR-β | ARAD_1K 4× | PSNR=40.46dB, SSIM=0.9605, SAM=1.18° | | MCNet-DL (unsup)| Urban 4× | mPSNR=26.69dB, mSAM=14.53°, ERGAS=7.60 | | RDN-DL (unsup) | Urban 4× | mPSNR=27.78dB, mSAM=12.14°, ERGAS=6.37 |

Loss functions typically combine per-pixel MSE or L1, possibly with spectral angle mapper, spatial–spectral gradient, or Huber losses. Custom regularization or auxiliary losses—e.g., sparse-spline penalties in KAN, total-variation, or hybrid image+wavelet terms—may be included.

6. Challenges, Limitations, and Future Research

HS-SISR is challenged by:

• Severe ill-posedness: especially in RGB-to-HS lifting; the spectral inverse problem is fundamentally underdetermined.
• Data scarcity and misalignment: real RGB-HS pairs and HR-LR registrations are rare or noisy; synthetic data is often required (Galliani et al., 2017, Xu et al., 30 Jan 2026).
• Sensor dependency: pretrained models are sensitive to the camera spectral response and operating band configuration (Zhang et al., 2021).
• Over-smoothing and spectral distortion at large upscaling: convolutional models may fail to maintain spectral fidelity at high scale factors (Muhammad et al., 6 May 2025, Karayaka et al., 10 Dec 2025).

Ongoing research explores:

• Transformer-based encoders/decoders for global spectral context (Ma et al., 2021)
• Plug-and-play physical priors and perceptual losses for better texture and spectral realism (Muhammad et al., 26 Sep 2025)
• Meta-learning and continuous-wavelength modeling for arbitrary band settings and cross-sensor generalization (Zhang et al., 2021)
• Test-time self-training and spectral-mixup data augmentation to mitigate domain shift and data scarcity (Li et al., 2024)
• End-to-end differentiable unmixing for unsupervised or self-supervised learning in the absence of annotation (Xu et al., 23 Jan 2026)

A plausible implication is that future HS-SISR frameworks will increasingly integrate physically informed priors, domain adaptation strategies, and hybrid explicit–implicit modeling to achieve robust generalization across diverse sensors and real-world scenarios.

7. Connections with Related Inverse Problems

HS-SISR is closely connected to other spectral and spatial super-resolution tasks:

• Spectral super-resolution (from RGB/multispectral to HSI) (Galliani et al., 2017, Yan et al., 2018)
• Spatial super-resolution for narrow-band signals (low-resolution HSI to HR-HSI) (Muhammad et al., 6 May 2025)
• Joint spatial-spectral SR (fusing HR-MSI with LR-HSI) (Li et al., 2024, Ma et al., 2021)
• Unmixing-based inverse methods, where the abundance estimation is itself a regularized inverse problem (Xu et al., 30 Jan 2026)
• Cross-modal and meta-learning SISR, enabling adaptation to arbitrary spectral settings (Zhang et al., 2021)

Through integration of deep learning, meta-learning, and physically motivated strategies, HS-SISR continues to evolve toward general, efficient, and data-efficient solutions.