PanTiny: Efficient Pan-Sharpening Framework

• The paper introduces PanTiny, a compact single-step pan-sharpening framework that fuses high-resolution panchromatic and low-resolution multispectral images with low computational cost.
• It employs a single-encoder transformer design with an Enhanced Conv fusion module, demonstrating competitive PSNR and QNR metrics versus larger models.
• The unified all-in-one training paradigm across WV2, WV3, and GF2 enhances cross-sensor generalization, making PanTiny practical for real-world remote-sensing applications.

PanTiny is a lightweight, single-step pan-sharpening framework proposed as a response to the trend toward increasingly large, dataset-specific pan-sharpening models that incur high computational cost and often generalize poorly to full-resolution or cross-satellite settings. It is defined by three coupled design choices: a compact single-encoder transformer architecture, an “all-in-one” or “multiple-in-one” training paradigm in which one shared model is trained jointly on WV2, WV3, and GF2, and a universal composite loss intended to improve not only PanTiny itself but a broader class of pan-sharpening models. In this formulation, PanTiny is less a single isolated architectural novelty than a system-level rethinking of efficiency, training regime, and loss design for pan-sharpening (Zhang et al., 20 Jul 2025).

1. Task setting and design rationale

Pan-sharpening is the task of fusing a high-resolution panchromatic image and a low-resolution multispectral image to produce a high-resolution multispectral image. The central difficulty is to inject spatial detail from the PAN image without corrupting the spectral fidelity of the multispectral signal. The paper situates this problem in remote-sensing applications such as land-cover analysis, environmental monitoring, and urban planning, and argues that recent progress has been distorted by a brute-force scaling trend: models have become larger, more computationally expensive, and more tightly specialized to individual satellite datasets, while generalization to full-resolution imagery has remained weak (Zhang et al., 20 Jul 2025).

Three practical issues motivate PanTiny. The first is efficiency: the paper explicitly criticizes CFDCNet for a computational cost of 55.73 G FLOPs at 128×128 resolution. The second is generalization to full-resolution imagery: reduced-resolution benchmark gains do not necessarily transfer to real full-resolution test conditions. The third is cross-satellite robustness: most prior systems follow a “one-dataset, one-model” paradigm, which makes deployment inconvenient and weakens transfer across sensors. The paper further argues that pan-sharpening datasets are relatively small, so increasing model size often promotes overfitting rather than transferable generalization. An appendix result sharpens this critique by showing that a model trained for just one epoch on a source dataset can achieve cross-domain results comparable to some purported “generalization” methods trained much longer, suggesting that prolonged single-dataset training can primarily reinforce source-domain bias rather than genuine robustness.

Within that framing, PanTiny is intended for practical deployment scenarios that require a single compact model with low inference cost, easier maintenance than multiple specialized models, and better behavior on full-resolution imagery. Its contribution is therefore as much about training philosophy and deployment form as about network topology.

2. Architecture and representational structure

PanTiny takes as input an upsampled multispectral image and a PAN image, and outputs a fused HRMS image. Its high-level pipeline has four stages: a single lightweight convolutional encoder for the MS input, a PAN-feature fusion module, a transformer body, and a final convolutional refinement layer (Zhang et al., 20 Jul 2025).

A defining architectural decision is the use of a single encoder rather than separate PAN and MS encoders. The upsampled MS image is first passed through a lightweight convolutional block to produce initial features, after which the PAN image is injected in feature space. The paper reports that this single-encoder design is both smaller and slightly better than a dual-encoder alternative: appendix variants m5 and m6 have 118.5K params and 64.3K params, respectively, while m6 achieves slightly higher performance on all three datasets, with WV2 41.06 vs 41.05, WV3 30.00 vs 29.89, and GF2 47.54 vs 47.45.

A second major decision is to avoid U-Net-style hierarchical downsampling. The adopted design is explicitly flat and full-resolution. In the appendix ablation, a model with 4 downsampling levels has 446.7K params and 14.64G FLOPs, a model with 2 downsampling levels has 121.2K params and 3.97G FLOPs, and the 0-downsampling model has 48.0K params and 1.57G FLOPs while remaining best or essentially tied in performance. This motivates PanTiny’s rejection of aggressive multi-scale down/up-sampling hierarchies.

The fusion block is also deliberately simple. The paper compares 1×1 Conv, Channel Attention, Gated Conv, DeepFusion from Pan-Mamba, and Enhanced Conv, selecting Enhanced Conv as the final design. Enhanced Conv consists of two consecutive 3×3 convolutional layers. In the reported full ablation, it uses 81.7K parameters and yields the best overall results: WV2 41.85 / 0.9696 / 0.0224, WV3 30.59 / 0.9238 / 0.0749, and GF2 48.61 / 0.9894 / 0.0095. The paper interprets the weaker performance of heavier fusion modules as an overfitting effect in the multi-dataset setting.

The transformer body uses standard Transformer blocks in Pre-LayerNorm form:

$X'_{l} = \text{CA}(\text{LN}(X_{l-1})) + X_{l-1}$

$X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$

where $\text{CA}$ is a Channel Attention module and $\text{GDFN}$ is a Gated-DConv Feed-Forward Network. Channel attention is performed across channels with

$$\text{Attention}(Q, K, V) = \text{Softmax}\big((Q_n K_n^T)\cdot \tau\big)V_n,$$

with $Q_n$ and $K_n$ L2-normalized and $\tau$ learnable. The feed-forward block is gated and uses depth-wise convolutions. The reconstruction head is intentionally minimal: a single convolutional layer. More elaborate refinement alternatives, including channel-attention refinement and larger-convolution refinement, were tested but did not show consistent benefit.

The paper does not provide a full layer-by-layer specification for the number of transformer blocks, embedding width, attention heads, or most kernel sizes. Those details are absent from the provided text and are not recoverable from the paper description alone.

3. Multiple-in-one unified training

A central conceptual contribution of PanTiny is its multiple-in-one or all-in-one training paradigm, in which a single set of weights is trained simultaneously on WV2, WV3, and GF2 (Zhang et al., 20 Jul 2025). The paper treats this as a direct alternative to the conventional one-dataset, one-model pattern.

This unified training is made possible by aligning the datasets to a common 4-band setup. The appendix states that WV2 and WV3 provide 8-band MS imagery, but are conventionally reduced to 4 bands for standard pan-sharpening tasks, while GF2 directly provides 4-band data. The training sets are then combined, and the same model weights are evaluated on each dataset. There is no mention of dataset-specific heads, prompts, adapters, or condition tokens; the model is fully shared.

The reported evidence shows that PanTiny is unusually robust under this joint regime. For PanTiny (Big), separate training yields WV2 42.16, WV3 30.61, and GF2 48.93, whereas all-in-one training yields WV2 41.85, WV3 30.59, and GF2 48.61. The degradation is therefore small. The paper contrasts this with larger specialized models that degrade more severely under all-in-one training; for example, Pan-Mamba on GF2 drops from 47.65 to 43.98.

The authors argue that this shared-weight exposure to multiple sensors improves robustness by discouraging sensor-specific memorization. That claim is most visible in full-resolution no-reference evaluation. On WV2, all-in-one training increases QNR for all reported models, including MSDCNN: 0.7683 → 0.8898, PNN: 0.7527 → 0.8844, PSCINN: 0.7904 → 0.8849, PanTiny Big: 0.7985 → 0.8793, and PanFlow: 0.7910 → 0.8900.

This unified regime does not, however, eliminate all cross-domain limitations. The appendix zero-shot test on Jilin-1, unseen during training, shows that all models remain weak. PanTiny Big reaches 23.10 PSNR, 0.5694 SSIM, and 0.0884 SAM, whereas PSCINN performs best with 27.90 PSNR, 0.8319 SSIM, and 0.0812 SAM. This indicates that all-in-one training improves robustness across the training sensors, but does not solve truly unseen-sensor generalization.

4. Universal composite loss and optimization

PanTiny is trained with a universal composite loss

$$L_{total} = \lambda_1 L_{1} + \lambda_2 L_{SSIM} + \lambda_3 L_{Focal},$$

with recommended weights

$$\lambda_1 = 1.5,\quad \lambda_2 = 4.0,\quad \lambda_3 = 1.5.$$

The $X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$0 term is actually the Charbonnier loss,

$X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$1

and the structural term uses

$X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$2

with

$X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$3

The third term is a regression-style focal loss intended to upweight hard pixels with larger errors. The paper’s printed formula for this term is corrupted in the provided text, so only its intended behavior can be stated with confidence (Zhang et al., 20 Jul 2025).

The loss ablations are important because the paper argues that loss design has been underexplored relative to architecture scaling. On PanTiny, L1 only performs markedly worse than the composite formulation; on GF2, it yields 45.42 PSNR. SSIM only is already strong at 47.21, equal weights (1,1,1) give 47.68, and the recommended weighting yields 48.29 or 48.61 depending on the model/table context reported in the paper. The appendix describes a two-stage search over the loss coefficients, ending at $X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$4, and the paper emphasizes that a relatively large SSIM weight is especially effective.

Optimization details are fully specified in the provided text. The framework is PyTorch, training uses a single NVIDIA RTX 4090, the optimizer is ADAM, the learning rate is $X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$5, betas are $X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$6, the scheduler is cosine annealing, training runs for 500 epochs, and the batch size is 16. Evaluation uses both reduced-resolution reference metrics—PSNR, SSIM, SAM, and ERGAS—and full-resolution no-reference metrics—$X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$7, $X_{l} = \text{GDFN}(\text{LN}(X'_{l})) + X'_{l}$8, and QNR. The paper states that standard protocols are followed, but patch size, augmentation strategy, exact dataset sampling ratio, and the precise focal-loss formula are not cleanly specified in the provided text.

5. Empirical performance and efficiency profile

The paper evaluates PanTiny on WV2, WV3, and GF2 in the all-in-one single-model setting, with direct comparison against PNN, PanNet, MSDCNN, PanFlow, PSCINN, Pan-Mamba, and CFDCNet (Zhang et al., 20 Jul 2025).

Model	Params	FLOPs
PanTiny Small	48.3K	1.58G
PanTiny Big	81.7K	2.68G
PanFlow	87.3K	2.86G
Pan-Mamba	488.8K	16.02G
CFDCNet	1700.8K	55.73G

In the main comparison table, PanTiny Big reports WV2 41.85 / 0.9696 / 0.0224, WV3 30.59 / 0.9238 / 0.0749, and GF2 48.61 / 0.9894 / 0.0095 for PSNR, SSIM, and SAM, respectively. PanTiny Small is already highly competitive with WV2 41.62, WV3 30.38, and GF2 48.16. Against CFDCNet, PanTiny Big is simultaneously smaller and stronger: 81.7K vs 1700.8K params, 2.68G vs 55.73G FLOPs, with higher PSNR on all three datasets. Against Pan-Mamba, the GF2 margin is especially pronounced: 48.61 vs 43.98. Against PanFlow, PanTiny Big also leads on reduced-resolution metrics.

The paper is careful, however, not to claim universal dominance on every evaluation axis. On WV2 full-resolution QNR, PanFlow slightly surpasses PanTiny: 0.8900 vs 0.8793 for PanTiny Big, while PanTiny Small reports 0.8751. This nuance is central to the paper’s framing: the main claim is not that PanTiny is best on every no-reference metric, but that principled design and all-in-one training improve the overall performance-efficiency balance and full-resolution robustness.

The scaling ablation reinforces the argument against brute-force enlargement. The paper reports four model sizes—Small: 48.3K, Big: 81.7K, Large Body: 172.4K, and Huge Body: 195.9K—with GF2 PSNR progressing only from 48.16 to 48.61, 48.75, and 48.85. The gain from increased capacity is therefore real but modest relative to cost, which the paper interprets as evidence of diminishing returns.

6. Interpretation, limitations, and place in the literature

PanTiny is positioned against several pan-sharpening traditions. Relative to traditional CS-based methods such as IHS, PCA, and Brovey-style approaches, it belongs to the modern learned-fusion paradigm rather than hand-designed spectral injection. Relative to MRA-based methods such as wavelet and SFIM-style methods, it replaces explicit multi-resolution decomposition with learned convolutional and transformer-based processing. Relative to recent deep pan-sharpening systems such as Pan-Mamba, PanFlow, PSCINN, and CFDCNet, its novelty lies less in inventing an exotic primitive than in combining a deliberately minimal architecture, unified multi-dataset training, and a universal composite loss (Zhang et al., 20 Jul 2025).

Several misconceptions are explicitly corrected by the empirical record. PanTiny is not simply “a smaller network”: its reported behavior depends strongly on the all-in-one training regime and on the composite loss, and the paper repeatedly argues that training philosophy and loss design are as important as the architecture itself. Nor is it a universal solution to cross-sensor generalization: Jilin-1 zero-shot performance remains poor for all models. It is also not best on every metric, as the WV2 QNR comparison with PanFlow shows.

The limitations are concrete. First, the paper text incompletely specifies the exact architecture, so reproducing the model from the paper alone is difficult without code. Second, zero-shot unseen-sensor generalization remains limited. Third, some important procedural details—patch size, augmentation, exact per-dataset sampling strategy, and the exact regression focal formula—are not cleanly specified in the provided text. These omissions restrict purely paper-based reproducibility. The released codebase partly addresses that issue, and the paper notes a unified, YAML-based experimental runner with hierarchical config inheritance at https://github.com/Zirconium233/PanTiny.

Taken together, PanTiny’s significance lies in reframing pan-sharpening as an efficiency-and-generalization problem rather than a benchmark-maximization problem. The architecture is intentionally compact, the training regime is intentionally shared across WV2, WV3, and GF2, and the loss is intentionally designed to transfer across models. The broader thesis advanced by the work is that principled engineering in model design, training paradigm, and objective function can outperform or rival much larger, sensor-specific pan-sharpening systems under realistic deployment constraints.