Swin2SR: Vision Transformer for SR

• Swin2SR is a vision transformer-based architecture that improves compressed image super-resolution and restoration through a robust three-stage pipeline.
• It integrates shallow feature extraction, deep residual SwinV2 transformer blocks, and pixel-shuffle upsampling to achieve detailed image reconstruction.
• Innovative loss functions and domain-specific extensions like CSwin2SR and Swin2-MoSE enable state-of-the-art performance in various low-level vision tasks.

Swin2SR is a vision transformer-based architecture for compressed image super-resolution (SR), restoration, and related low-level vision tasks. Built atop the SwinV2 transformer, Swin2SR addresses major challenges in training deep vision transformers for SR, such as instability, resolution transferability, and high data requirements. It has catalyzed subsequent innovations including closed-loop extensions, adaptations for medical imaging and remote sensing, and the integration of mixture-of-experts strategies.

1. Architectural Foundations

Swin2SR is structured into a three-stage pipeline: shallow feature extraction, deep hierarchical representation learning with residual SwinV2 transformer blocks (RSTBs), and final image reconstruction with upsampling.

• Shallow feature extraction: A single $3 \times 3$ convolution is employed to capture low-frequency content from the (often compressed or degraded) input.
• Deep feature extraction: A cascade of $N$ RSTBs, each containing $m$ SwinV2 transformer layers, a $3 \times 3$ convolutional tail, long skip connection, and local residual paths. The default configuration uses $N=6$, $m=6$, channel dimension $C=180$, window size $M=8$, and $h=6$ attention heads.
• Reconstruction head: Final reconstruction employs a convolutional layer followed by pixel-shuffle upsampling, enabling resolution enhancement with minimal artifact introduction.

Key architectural enhancements relative to SwinIR include:

• Window-based multi-head self-attention (W-MSA): Restricts attention computation to local non-overlapping windows, reducing computational cost and memory requirements from global self-attention.
• Scaled cosine attention and log-spaced continuous relative position bias: Improves deep model stability and cross-resolution generalization, essential for SR scenarios where train/test resolutions may not match.
• Post-layer normalization: Applied after attention/MLP modules, further contributing to stable extremely deep transformer training (Conde et al., 2022).

2. Training Losses and Optimization

Swin2SR is optimized primarily with $\ell_1$ loss between predicted output and ground-truth HR image:

$N$0

For compressed-input SR, auxiliary objectives are incorporated:

• Downsample consistency loss:

$N$1

where $N$2 denotes a downsampling operator. This enforces consistency at multiple image scales.

• High-frequency reconstruction loss:

$N$3

with $N$4 as a Gaussian blur. This targets retransmission of high-frequency details lost due to compression and degradation.

The final combined loss is:

$N$5

with typical choice $N$6 (Conde et al., 2022).

3. Representative Applications and Empirical Results

Swin2SR is validated on compressed image super-resolution, classical/lightweight single-image SR, and compression artifacts removal. Notable empirical highlights:

• Compressed SR (DIV2K, AIM 2022): Achieves PSNR 23.4033–23.4946 dB (JPEG Q=10, $N$7 upsampling) with top-5 challenge placement, and does so with lower computational cost than most CNN and transformer competitors.
• JPEG artifact removal and classical SR: Demonstrates state-of-the-art or highly competitive results compared to leading alternatives such as SwinIR, RNAN, RCAN, and IPT, often matching or surpassing their performance with fewer training iterations or parameters.
• Ablations: Marginal but reliable gains are observed from auxiliary losses and self-ensemble strategies. For example, in compressed-input SR on DIV2K validation, addition of auxiliary and high-frequency losses yields a cumulative $N$8+0.01 dB PSNR gain (Conde et al., 2022).

4. Derivatives and Extensions

The modular backbone of Swin2SR has enabled both general and domain-specific extensions:

4.1. Circular Swin2SR (CSwin2SR)

CSwin2SR introduces a closed-loop negative feedback mechanism, building upon the open-loop Swin2SR. It iteratively re-injects the current HR estimate via downsampling and recompression, computing the error against the initial SR output and updating the reconstruction:

$N$9

with $m$0 and typically $m$1 iterations. This framework enables systematic iterative refinement and feature reuse, yielding up to +0.18 dB PSNR and +0.01 SSIM over Swin2SR, and larger local improvements on critical image regions (Li et al., 2023).

4.2. Domain-Specific Adaptations

Remote Sensing (Swin2-MoSE):

• Incorporates mixture-of-experts (MoE) layers with a “Smart Merger” in place of FFN, and jointly exploits per-head and per-channel positional encodings.
• Optimization uses a weighted sum of normalized cross-correlation (NCC) and SSIM losses, directly addressing MSE-induced over-smoothing.
• Provides consistent improvements (up to +0.96 dB PSNR) over Swin2SR in 2x/3x/4x SR on Sen2Venμs and OLI2MSI datasets (Rossi et al., 2024).

Medical Imaging (Complex Swin Transformer):

• Features separate processing streams for magnitude and phase, followed by a shared Swin2SR-style backbone. Final complex recomposition reconstructs data for multi-echo MRI (SMWI).
• Trained and tested on 214 SMWI scans, achieving SSIM ≈ 0.91 and MSE ≈ 0.064 on $m$2 inputs (Usman et al., 21 Dec 2025).

5. Quantitative Benchmarks

Performance overview

Task	Dataset	Metric	Swin2SR	Derivative	Notable Results
$m$3 Comp. SR	DIV2K (Q=10)	PSNR (dB)	23.4033–23.4946	CSwin2SR	+0.18 dB over Swin2SR (Li et al., 2023)
$m$4 SR	Set5	PSNR (dB)	38.43	Swin2-MoSE	+0.43 to +0.96 over Swin2SR (Rossi et al., 2024)
SMWI SR	SMWI	SSIM	—	Complex Swin	0.9116 for $m$5 k-space (Usman et al., 21 Dec 2025)

Further, CSwin2SR outperforms Swin2SR particularly in nearest-neighbor downsampling scenarios (e.g., $m$6PSNR = +0.28 dB on Set5), and Swin2-MoSE outperforms all Swin-based models in remote sensing benchmarks.

6. Algorithmic Innovations and Impact

Swin2SR demonstrates that advanced transformer design, when properly adapted for low-level vision, can match or surpass the best CNN-based super-resolution methods with modest data requirements and without large-scale pretraining. Its design principles—windowed attention, log-bias encoding, post-norm, and enhanced loss construction—have been inherited and extended in subsequent work spanning multiple imaging disciplines and massive remote sensing scenes.

The closed-loop feedback (Li et al., 2023), MoE feedforwards (Rossi et al., 2024), and complex-valued stream strategies (Usman et al., 21 Dec 2025) all illustrate the flexibility and extensibility of Swin2SR as a backbone for both general and highly specialized super-resolution contexts.

7. Implementation and Availability

Swin2SR and derivatives are implemented in PyTorch, with open-source releases for both image restoration and remote sensing variants (Conde et al., 2022, Rossi et al., 2024). Training typically employs Adam optimizer ($m$7, $m$8), window size $m$9, and batch sizes of 8–16. Hardware is generally NVIDIA A100 or equivalent GPUs. For CSwin2SR, the feedback loop uses $3 \times 3$0, $3 \times 3$1 iterations. No adversarial or perceptual losses are required; training follows the original Swin2SR regime. Code and pretrained models are accessible for adaptation and benchmarking across domains.

---

Swin2SR exemplifies the convergence of hierarchical transformers with principled architectural and loss-based regularization for SR, establishing a blueprint for future domain-adaptive and feedback-driven image restoration pipelines (Conde et al., 2022, Li et al., 2023, Rossi et al., 2024, Usman et al., 21 Dec 2025).